TOOL-BASED TRANSFORMATIONS

Dr. Tom Froese

IIMAS-UNAM

Toward a new cognitive science• In cognitive science we are seeing a paradigm shift. Gone

is the idea that “mind = digital computer”.

• Instead, it has been found that mind is essentially:• Embodied• Embedded• Extended• Experiential

• And there is growing evidence that mind is also:• Technological

The performance artist Stelarc

Mind-body problem

Livingbeing

Physical body

Matter Mind

???

Descartes (1596-1650)

Traditional dualism: the body is a physical mechanism like any other in the world.

The relationship between consciousness and matter remains a complete mystery.

How to explain mental transformations caused by tool use?Froese (2011)

The mind-body-body problem

Livingbeing

Lived body

Physical body

Matter Mind

???

Mind is embodied. I experience the world by

means of my phenomenally lived body (Leib).

Husserl (1859-1938)

Descartes neglected embodiment.

The body is not just mechanism, it is our lived flesh. We are our body.

Froese (2011)

Life-mind continuity

Life Lived body

Living body

Matter Mind

Thompson (2007)

Phenomenology: mind is embodied in a living body.

The relationship between consciousness and matter can be understood through the phenomenon of life.

Froese (2011)

Froese and Di Paolo (2011)

Dynamical approach to cognition

Beer (2000)

• The brain is embodied in a biological body.

• The body is situated in an environment.

• Behavior is an emergent property of a brain-body-environment system.

• What happens if we place an interface between the body and its environment?

Traditional interface design

• Cognitivist view of tools:• Subject and tool are

independent• The tool is just another

object in the world• Sensorimotor interaction

loops are ignored• Symbolic information is

given precedence over direct perceptual experience

Froese, McGann, Bigge, Spiers and Seth (2012a)

EDSAC Computer, 1949, Cambridge Uni.

Enactive interface design

• Embodied view of tools:• Subject and tool are

interdependent• The tool is an interface• Interfaces mediate

sensorimotor interaction• Continuous interaction

shapes user experience

Froese et al. (2012a)

Paradigmatic example: sensory substitution interfacesTactile-visual substitution system (TVSS)

Tools for studying the embodied mind

“Artificial embodiment”• If mind is rooted in a living body, then this means serious

trouble for replicating our minds artificially.• Bad news for traditional AI and robotics! (Froese and Ziemke 2009)

• If mind is embodied in life, then this means profound new opportunities for transforming our minds artificially.• Good news for human-computer interaction (HCI)!• Clark (2003). Natural-Born Cyborgs: Minds, Technologies, and the

Future of Human Intelligence

• If mind is embodied, and using technology transforms our embodiment, then technology can transform our mind!• Useful scientific methodology! (Froese et al. 2012b)

Effects of tool-use• Psychologists have been busy documenting the various

ways in which the use of tools affects the users.

• Changes in body schema (living embodiment)• Changes in perceived space• Changes in body image (lived embodiment)

• Body schema / body image? (Gallagher 1986)

• Usually, but not always, such tool use based changes happen to various extents at the same time.• Effect of modulation, not control.

Changes in body schema• There is well-known evidence that use of elongated tools can lead to

changes of the neural organization related to the body schema.

• For example, distant object manipulation with a rake leads to an extension of the neural body schema for the arm (Iriki et al. 1996).

• Associated visual receptive fields are also extended.• Tool-based modulation of sensorimotor loop.

• This only occurs after intentional usage of the tool, not during passive holding.

Maravita and Iriki (2004)

Maravita and Iriki (2004)

Iriki et al. (2001)

Changes in perceptual neglect of space

• Berti and Frassinetti (2000) tested a patient who showed perceptual neglect in near space during a similar line bisection task.

• The patient showed no neglect in far space when using a laser pointer to do the task.

• The patient also showed neglect in far space when using a long stick to bisect the line.

Changes in perceived space• When participants are asked to

indicate the middle of a line using a laser pointer, there is a systematic rightward deviation as the line is placed further and further away.

• Longo and Lourenco (2006) found that using an elongated stick in the same task eliminated this bias.

• Tool use may be extending the range of what is seen as ‘near space’ according to reachability.

Changes in perceived space• Witt et al. (2005) manipulated reachability with an

elongated tool and asked participants for estimates of a target object’s distance.

• They showed that tool-users perceived shorter distances to target objects that were beyond normal reach, but within reach with the tool.

• They also found that manipulating reachability only influenced perceived distance when the perceiver actually intended to reach the object.

Witt et al. (2005)

Act

ive

reac

hing

con

ditio

n

Witt et al. (2005)

Pas

sive

hol

ding

con

ditio

n

Changes in body image• Cardinali et al. (2009) found that using a mechanical

grabber that extends the arm alters the kinematics of subsequent free-hand grasping (body schema).

• Furthermore, this effect is related to an increase of the perceived length of the arm (body image)

Tool-use modifies movement kinematics and somatosensory morphology.

Cardinali et al. (2009)

Changes are task dependent• Farnè, Iriki and Làdavas (2005) showed that these kinds

of changes critically depend upon active tool use. • They are not found after passive exposure to a hand-tool body

configuration.

• They demonstrated that the elongation extent of near hand space after tool-use is related to the operationally effective length of the tool, and not simply its absolute length.

Changes in body ownership• The rubber hand illusion (Botvinick & Cohen 1998)

Changes in body ownership

The body swapping illusion (Petkova & Ehrsson 2008)

Body swapping illusion

Petkova and Ehrsson (2008)

Substitutional reality system

Suzuki, Wakisaka and Fujii (2012)

Suzuki, Wakisaka and Fujii (2012)

Conclusions• If the hypothesis of life-mind continuity is correct, then the

possibilities of replicating human intelligence in robots and other technological systems is limited – no life, no mind!

• But our embodied mind presents us with open-ended possibilities for self-transformation – by designing new technological interfaces that shape our existence.

• The goal should be to extend human capacities with technological interfaces, and not to externalize our existing capacities to make machines more autonomous.

Homework• Try out the rubber hand illusion!

• Finish reading:

• Froese, T., McGann, M., Bigge, W., Spiers, A., & Seth, A. K. (2012a). The Enactive Torch: A new tool for the science of perception. IEEE Transactions on Haptics, 5(4), 365-375.

References• Beer, R. D. (2000). Dynamical approaches to cognitive science. Trends in Cognitive Sciences, 4(3), 91-99.

• Berti, A., & Frassinetti, F. (2000). When far becomes near: Remapping of space by tool use. Journal of Cognitive Neuroscience, 12(3), 415-420.

• Botvinick, M., & Cohen, J. (1998). Rubber hands 'feel' touch that eyes see. Nature, 391, 756.

• Cardinali, L., Frassinetti, F., Brozzoli, C., Urquizar, C., Roy, A. C., & Farnè, A. (2009). Tool-use induces morphological updating of the body schema. Current Biology, 19(12), R478-R479.

• Clark, A. (2003). Natural-Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence. New York, NY: Oxford University Press.

• Ehrsson, H. H. (2007). The experimental induction of out-of-body experiences. Science, 317, 1048.

• Farnè, A., Iriki, A., & Làdavas, E. (2005). Shaping multisensory action-space with tools: Evidence from patients with cross-modal extinction. Neuropsychologia, 43, 238-248

• Froese, T. (2011). From second-order cybernetics to enactive cognitive science: Varela's turn from epistemology to phenomenology. Systems Research and Behavioral Science, 28, 631-645.

• Froese, T., & Di Paolo, E. A. (2011). The enactive approach: Theoretical sketches from cell to society. Pragmatics & Cognition, 19(1), 1-36.

• Froese, T., McGann, M., Bigge, W., Spiers, A., & Seth, A. K. (2012a). The Enactive Torch: A new tool for the science of perception. IEEE Transactions on Haptics, 5(4), 365-375.

• Froese, T., Suzuki, K., Ogai, Y., & Ikegami, T. (2012b). Using human-computer interfaces to investigate 'mind-as-it-could-be' from the first-person perspective. Cognitive Computation, 4(3), 365-382.

• Gallagher, S. (1986). Body image and body schema: A conceptual clarification. The Journal of Mind and Behavior, 7(4), 541-554.

References• Iriki, A., Tanaka, M., & Iwamura, Y. (1996). Coding of modified body schema during tool

use by macaque postcentral neurones. NeuroReport, 7(4), 2325-2330.

• Iriki, A., Tanaka, M., Obayashi, S., & Iwamura, Y. (2001). Self-images in the video monitor coded by monkey intraparietal neurons. Neuroscience Research, 40, 163-173

• Longo, M. R., & Lourenco, S. F. (2006). On the nature of near space: Effects of tool use and the transition to far space. Neuropsychologia, 44, 977-981.

• Petkova, V. I., & Ehrsson, H. H. (2008). If I were you: Perceptual illusion of body swapping. PLoS ONE, 3(12), e3832. doi: 10.1371/journal.pone.0003832

• Suzuki, K., Wakisaka, S., & Fujii, N. (2012). Substitutional reality system: A novel experimental platform for experiencing alternative reality. Scientific Reports, 2(459). doi: 10.1038/srep00459

• Thompson, E. (2007). Mind in Life: Biology, Phenomenology, and the Sciences of Mind. Cambridge, MA: Harvard University Press.

• Maravita, A., & Iriki, A. (2004). Tools for the body (schema). Trends in Cognitive Sciences, 8(2), 79-86.

• Witt, J. K., Proffitt, D. R., & Epstein, W. (2005). Tool use affects perceived distance, but only when you intend to use it. Journal of Experimental Psychology: Human Perception and Performance, 31(5), 880-888.