Redefining Humans Ted Krueger In Press: unpublished draft Images occur at the end of the text. This paper developed out of a presentation by the same name that has been given at a number of universities and transformed to fit a variety of contexts. When given to an audience consisting substantially of undergraduate designers it begins by asking them to imagine the inhabitation or experience of their most recent design project. By means of a series of simple questions1, it highlights the fact that many, if not most, inexperienced designers rely exclusively on first-person visual imagination. I expect that many experienced designers do the same. What assumptions are buried in that first person viewpoint and how do they affect the design solution? When considering the ‘redefinition’ of the human it would seem reasonable to ask two questions. The first is ‘what is the current definition of human?’ and the second is ‘why change it?’ Brooks (1991) in ‘Intelligence without Reason’ defines intelligence as “what humans do pretty much all of the time”. This statement was a provocation to those researchers in artificial intelligence who had spent considerable effort in developing a carefully crafted definition that could be operationalized in the activities of their laboratories. This seemingly offhanded attitude, however, was specifically chosen for a perspective that privileged ‘behavior-in-context’ over a universal definition. Following this approach, the first question might be answered ‘Human is what you think it is pretty much all the time’. This answer has less to do with wishing to avoid the inevitable entanglements that result from an attempting a universal and timeless answer but more, the desire to indicate that the definition of interest is not the one that I could concoct, but is the one that you, as a designer, might use in your work.2 This model of the human is commonly a hybrid of introspection, experience and education. Typically, we understand ourselves as a singular, cohesive and enduring relatively unchanged through time; an individual operating within an objective social and physical context. Models and Frameworks The model of the human that we use to design determines the kinds of problems as well as the field potential solutions that are considered. The framework within which design is undertaken allows only limited access to the solution space of a particular problem. Here, solution space refers to the set of all possible solutions to a given problem. Solution sets for arithmetic problems may be singular but for most design problems are vast. In reference to a specific dimension of the problem, some solutions may be better than others. A solution space may resemble a landscape in which better solutions, with respect to that dimension, are hills and poorer ones, valleys. Landscapes perhaps, but discontinuous ones, there is no requirement that this metaphoric surface is continuous - holes, discontinuities and fragments occur. Problem solving strategies vary in the ways in which they sample or search through this space of potential solutions and the portions of the solution space to which they have access. This issue of accessibility can be illustrated by reference to a study conducted by Thompson (1996) involving the configuration of a field programmable gate array (FPGA) to detect a change in frequencies. An FPGA is a semiconductor chip containing an array of standard logic gates that can be connected together to form complex circuitry under the control of software running on a host computer. This allows the generic circuit to become configured for any number of specific tasks. Thompson uses this configurability to produce a frequency discriminating circuit by means of a genetic algorithm. The algorithm is allowed to configure only one hundred of the gates and is not given access to timing circuitry to coordinate between them. Yet, after 5000 generations a stable solution is found. (figure 1)

An investigation of the resulting circuit was undertaken revealing many gates that are not connected and therefore do not contribute to the circuit. Several that are shown toned on figure 2 are not connected to the circuit directly, but the circuit will not operate correctly if they are eliminated3. The operating principles of the circuit are not clear. It does not follow a digital logic and there is no clock cycle but it relies instead on subtle interactions between the gates, and between the FPGA and the host. It is very sensitive to changes in its environment. It is notable that this circuit has been developed on a substrate specifically designed to support digital operations and yet operates entirely without them. The fact that the circuit operation is very difficult, if not impossible, to describe with precision is not an attempt to conjure up a neo-vitalist spook ex machnia. The circuit operates as an analog device completely within the physics of semiconductors. The point of referencing this research is not to argue for the superiority of either the designed or automated process, but rather to consider the relationship between the subsets of the solution space that are sampled by human designers as opposed to those that are sampled by the genetic algorithm. Imagine assigning Thompson’s frequency discrimination task to an engineer. What is the probability that the algorithm’s solution will be produced? While a similar, but opposite consideration could be made for the operation of the algorithm. Thompson, et al (2000) suggests that the genetic algorithm searches solution spaces that are not available to human designers. ‘On the Design of Organisms’ (Krueger 2000b) reaches the same conclusion regarding biological processes and human design efforts and suggests that current design practice is methodologically inadequate to the task of designing the biological because it cannot access the appropriate regions of the solution space. It is reasonable to assume that Thompson’s is the first non-digital circuitry to be run on these chips. The ‘digital’ is a conceptual organization that operates completely in the realm of human culture not within the physics of the chip itself. The operation of the circuit is digital because we choose to think of it in that way and to work with the material within these self-imposed constraints. That there is no inherent ‘digital’ is proven by the results of the experiment as well as by the extreme efficiency of those results. In part, non-digital design is ‘unthinkable’ in this context because the absence of viable analog design techniques insures that the human cost of developing the design would easily outstrip the cost of circuitry that could be saved in the process. But the analog approach is also invisible because the non-digital aspects of the circuit are understood as problems to be overcome rather than opportunities to be exploited even though all circuit designers are aware that they exist. In a similar way, the glass used for fiberoptic communications was specially developed to be immune to environmental conditions that caused distortions of the signals. However, these same distortions can be read to yield information about the kinds of conditions that cause them making them useful in sensing applications. Fiberoptics embedded in structural components have a wide range of applications. (Krueger 1999). We choose to see opportunities and to deploy efforts and resources based on the frameworks that we employ. One person’s noise is another’s signal. Agre (1995) relates the ways in which technological development in Artificial Intelligence research has been hampered by contradictions inherent in its philosophical foundations. He notes that paradigmatic formulations consolidate an approach to the problems confronting a discipline at a particular point in time. This coherence allows research to proceed without a reconsideration of first principles, but it also circumscribes both the questions that are allowed to be asked and the methodologies that are available for their answering. Agre considers these codifications to be an institutionalized form of forgetting - a method of erasing competing and anomalous alternative formulations. His work is related to that of Thompson in that Agre shows that the framework within which work is undertaken restricts the visibility of potential solutions and thereby circumscribes those portions of the solution space that may be searched to ones of a particular level of effectiveness. The philosophical underpinning of a discipline sets limits on what can be achieved.

I would like apply this understanding to the frameworks that we use when we design, specifically to our intuitions about the ‘human’. In this paper, a revised understanding of the relationship of humans and environments will be drawn from contemporary research in wide variety disciplines including anthropology, cognitive science, perceptual psychology, physiology, and biology. This research together suggests that the image of ourselves as singular, cohesive and enduring entities given to us by introspection and by our experience is both inadequate and inaccurate. Importantly for the fields of design, traditional models underplay the dependencies that we have developed with our environments and in doing so undervalue the role of the design professions in crafting those environments. This alternative image of ourselves not only suggests a more central role for design, but has some direct suggestions for the kinds of design projects that might be undertaken. Adaptations to Microgravity Some of what we consider to be our inherent nature comes as the result of observation that has been undertaken in a remarkably uniform physical environment. When an altered physics is inhabited, in orbiting, planetary or interplanetary conditions, the human body rapidly adapts to the new. Bone de-densification and muscle atrophy are two of the many pervasive changes that occur. Simon (1985) notes that the complexity of an ant’s path on the beach is not due to the complexity of the ant but to that of the environment. With a similar logic, I would suggest that the constancy that we experience may be less inherent in us than due to the constancy in our environment. It is the adaptability that is fundamental and continuous. Under terrestrial conditions, bones are in a constant process of restructuring. Bones react to physical strain in a manner similar to a piezo-electric ceramic. The piezo-electric effect occurs when an asymmetric crystal emits a voltage under strain4. It is supposed that bone works in this way, emitting a signal that causes building at points of strain. Building continues until the strain is reduced, while unstrained bone is lightened until forces increase. In ‘Growth and Form’, D’Arcy Thompson (1917) shows that the section through a femur resulting from this dialogue between force and material is directly related the structural diagram of the forces acting on it (fig 3). In microgravity, the body quickly adapts to the altered physical conditions. Bones that were normally stressed by gravity loads begin to loose mass by as much as 1-2% per month. The excess calcium is excreted, resulting in an increased incidence of kidney stones. This radical transformation is a hypogravity-induced osteoporosis. Muscles react in a similar way. Extensors are more affected than flexors as they carry most of the gravity load on Earth (Unterman et al 2002). In microgravity, astronauts float and so move from place to place by pulling themselves along on or pushing off from the many handrails provided for this purpose. During transit, objects may be held between the knees. Effectively, the arms and legs have traded function in this respect. In these conditions, upper body strength is maintained while lower body muscle mass decreases as much as 11% per month while a rigorous exercise schedule is being maintained5. While these changes are rapid and pervasive, they are not unique to orbiting environments. By unloading gravity from the body, such as by continuous bed rest, much the same effects can be produced. Confining a subject to bed rest is, in fact, a standard experimental condition for simulating certain aspects of weightlessness (Fitts et al. 2000). Muscle mass may built up by simulating hypergravity at any neighborhood gym. Age related osteoporosis is common. Rather than assuming our own constancy, we should instinctively accept physiological transformation as we live it in the historical trajectory of our lives. The speed at which transformation occurs in micro-gravity allows it to be more easily identified. The adaptations rendered visible by immersion in microgravity show that we swiftly, even aggressively, conform to the conditions in which we find ourselves. These physiological adaptations are automatic responses to the new conditions. They are not a deterioration of capability unless one references the intent to return to earth. Of course, this information is not available to the physiology which deals only and directly with the immediate physical conditions.

What we intuit to be our intrinsic nature is only our nature in this world, under the conditions with which we have had experience. With a sample size of one, we should make no conclusive judgments about our fundamental nature or our capabilities should conditions change. The exploration of space will be accompanied by many discoveries - its permanent inhabitation by the discovery of other expressions of what is human. That may be the real prize. Physiological adaptations to microgravity are not limited to bones and muscles, but these serve to illustrate the kinds of processes that occur. Perception is also affected by microgravity and is recognized as a serious design consideration that affects orientation, navigation, and communication. Initial exposure to microgravity often results in a condition called space sickness, a nausea caused by the decoupling of visual, vestibular and proprioceptive sensations. With the radical reduction in gravity, the vestibular apparatus no longer plays a primary role in orientation. Similarly, proprioception due to the tug of gravity on muscle and joint and the working against gravity that many movements imply is eliminated. On earth, our sense of orientation depends on the synthesis of these in correlations that form, simultaneously, the constancy of the self and the stability of the world we inhabit. When the fusion of these perceptions can no longer be maintained, movements in the visual can not be assigned to either the self or the environment. A physical sickness results. While some level of adaptation to space sickness is typically obtained after several days, many astronauts experience much the same distress during the first days on Earth following a mission. On orbit, freed from the global organization imposed by the gravitational vector, orientation tends to depend much more on purely visual cues and for many the orientation in space becomes centered on the bilateral axial orientation of the body itself.6 On Earth, simulator sickness occurs for similar reasons. Imperfections in interface technology that introduce lags, distortions or visual cues that do not match vestibular and proprioceptive information effectively disintegrate perceptual fusion. In some of the early days of flight simulation pilots were forbidden flight for many hours after a simulation because ‘flash-backs’ and disorientation persisted. Simulations and virtual environments represent an altered physics which can be, in some sense, inhabited, and to which adaptation might be developed. However, the timeframes in which digital environments are inhabited generally do not permit several days of adjustment. Krueger(2000a) notes that issues of orientation within virtual environments are the subject of research into the ability to navigate and to work effectively in the digital realm as well as a concern in the design of orbiting environments. There is often a desire to impose an architectural trope on the virtual space as a means of organizing and orienting the activities that take place there. This imposition effectively reintroduces a local visual gravity and allows the body image to re inhabit a normative spatial continuum. The benefits of immediate and simple orientation are here exchanged for the potential benefits of exploiting the non-gravitational virtual world. Based on experiences in Skylab (figure 4), the International Space Station has a local visual gravity built into the design of each nodule (figure 5). The ISS is organized as a rectangular Cartesian lattice of corridors having a square cross-section with chamfered corners. Racks of experiments face each other across the ‘sides’ of the corridor. The ‘ceiling’ contains storage and the ‘floor’ life-support system racks. The diagonals near the ceiling contain lighting strips and those near the floor’ contain power and utility connections. Astronauts in each module work within a common visual frame of reference. Comparing the artists rendering with actual conditions (figure 6) gives some justification for some level of imposed organization. But understanding and providing for the need for orientation may not require the reassertion of global consistency. Devices that give provisional organization to relevant aspects of the environment and its inhabitants may work just as effectively while still allowing the freedom of movement to engender flexibility in the organization of the environment at the larger scales. The correspondences between the microgravity and the virtual suggest that designers of each type of space should become familiar with the experiments and experiences of the other. In microgravity, astronauts are likely to meet each other in arbitrary orientations. This has an effect on recognition and communication, especially on those nonverbal qualifications to

language given by nuances of facial expression. Initially, it may not seem difficult to recognize an inverted face (figure 7). The comparison here should not be between the face on the left and that on the right as inverting the image will prove. The inverted eyes and mouth, adapted from Rock (1984), generally come as a surprise. In addition, one astronaut encountering another in an inverted orientation often perceives him/herself to be ‘upside down’. A common local orientation facilitates social discourse and collaborative work. This can be seen in a series of student design projects included with this essay. The design of work surfaces for the International Space Station requires thought about the range of functions provided by common furniture. In microgravity, one has no need of a table to support objects at a convenient height. The work surface instead becomes understood as catalyst for cognitive organization (Kirsh 1995) referenced to the individual rather than to specific fixed locations in the craft. The furniture becomes wearable. The linking of individual surfaces provides a common reference plane that facilitates social interaction and communication. As a pedagogical exercise this has value for designers as it requires them to come to terms with aspects of the environment that are masked by the simple answers provided by the conventional solution of a rectangle on four legs resting on the floor. Microgravity in this case, makes visible relations of humans and environments that occur in common experience but are infrequently perceived. Written communication is a cultural product developed in gravity and includes a common standard orientation as a tacit assumption as it is not only oriented but directionally sequenced. Nonstandard orientations slow reading and introduce errors (figure 8). Relationships between characters such as and < L and 7 >, < N and Z>, < u and n>, < , and ‘ >, < i and ! >, < 6 and 9 >, or < p, d, q, and b > would be design flaws in written communication developed for orientation independent reading. Alteration of fundamental aspects of environments requires adaptive physiological and perceptual responses. It also illuminates aspects of the environment, such as written communication, recognition of faces, and the reading of expressions that has undergone an intrinsic adaptation to the former condition. While these short paragraphs are far from a comprehensive introduction to the relationship of humans to novel physical conditions, they should serve to illustrate that there is a profound interdependence between humans and environments. Plasticity and perceptual adaptation The dramatic perceptual adaptation that is quickly achieved during the first days on orbit is related to perceptual adaptations that can be experienced on earth. There is a long history of perceptual experiments involving altered vision by means of specially constructed glasses or goggles such as those of Stratton and Gibson in the United States, the extensive work by Kohler and Erismann at Innsbruck (Kohler 1964) and continuing to the present in the work of Sekiyama and colleagues. The experiments test adaptation to systematic alterations of the visual field and the simultaneous alteration in the relationship between movement and vision. Frequently, they result in the characteristic nausea before adaptation occurs. In many cases, the subject adapts to the new input modes and is able to function. Reviewing a series of experiments, Kohler (1962) notes a remarkable ability of vision to adapt to certain distortions introduced by experimental goggles. Wedge shaped prismatic glasses present images in which straight lines appear curved. The geometric distortions resulting from head movements in conjunction with movements of the eyes are complex, making it seem as if the objects are elastically deforming. Adaptation to these conditions occur in several weeks and the world appears both straight and stable once again. The wedged prisms also cause color fringes to appear because shorter frequencies are displaced more than longer ones. Adaptation to color displacements takes several days. Goggles have been used that invert the field of view, reverse it left to right and color one side differentially from the other. In all cases of systematic variation, adaptation occurs. Kohler describes the goggles as ‘breaking down’ the established modes of perception so that new functions can be learned. Kohler asserts that the recalibration of seeing to the new conditions provided by the goggles is not a novelty brought about by the experimental

conditions but an expression of the innate process of adaptation that is in operation in all seeing under normal conditions. These adjustments indicate that the specific properties of the sensory apparatus do not completely specify the perception; rather, it is the nature of the patterns that are produced that determines it. Held (1965) demonstrates that the development of perceptual adaptation is dependant upon an active engagement with the environment and requires that other sensorimotor systems come into play. The relationship between vision and movement is complex. Many would say, based on examples of visual adaptation, that proprioception grounds vision. But, proprioception itself is subject to instability. Wann and Inbriham (1992) documented a distinct drift in the perceived location of an arm when it could not be seen by the subject during their experiments. In the absence of vision, limb location is not completely specified. The finding is perhaps surprising in light of our experience of stability and cohesion among the senses. How could the sloppiness described by proprioceptive drift or the constant adaptation of vision be reconciled with perceived constancy and integration? Those experienced in architectural practices will recognize that precision components brought together in an absolute coordinate system are no guarantee of an integrated assembly. Because building sites are complex and contingent, intelligently considered points of adjustability and mutual coordination between components is a more effective strategy. The coordination between senses works in an analogous way. Clearly, in childhood when the length of a limb is constantly changing relationships between vision and hand location given by proprioception would also continuously change. A method of constant recalibration would be most efficient. A relationship that has the tendency to be in drift, but dominated by mutual recalibration will stay in alignment. In this way, the relationship between our integrated experience and the contingent nature of the senses can be understood not as paradox but as method. Adjustment in the interpretation and coordination between sense data might be considered a change in the way this information is processed - a form of learning, perhaps. In the popular understanding, the Cartesian duality - the mind/body split – suggests that this is a change in the ‘software’ not the ‘hardware’7. Nerves might be likened to wires in a circuit laid down according to genetic wiring diagram. The information flows from the eye via the optic nerve to the visual cortex where it is then interpreted. But, there are growing indications that this is not an accurate interpretation. Nerves are not wires (Bach-y-rita 1972). The hemispheric specialization of the brain, in which images are processed on the right and language and analysis on the left has been challenged by a new form and, therefore, understanding of language. Sacks (1989) reviews the development and subsequent linguistic analysis of American Sign Language (ASL). Despite official efforts to make the gestures (signs) used for communication by the deaf conform to the structure of English as a spoken and written language, ASL appeared in the 20th century as a fluid and effective means of communication. Its roots are in French sign languages of the 18th and 19th centuries. It’s recognition as a true grammatical and syntactic language was slowed by the fact that it is founded on entirely different principles than spoken and written languages. William Stokoe recognized that its structure is spatiotemporal. Neurological studies reported by Sacks indicate that ASL is processed in left hemisphere language area in a completely different way than simple gestures which remain on the right. Sacks cites this as conclusive evidence that ASL is a language. It is also evidence that the routing and processing of neural signals is not determined by the structure of the signals nor the biological transducer that produce nor the nerves that carry them. Animal studies suggest a similar conclusion derived from very different methods. Sur and colleagues at MIT have rewired the optic nerve of very young ferrets from the visual into the auditory cortex and confirmed through behavioral studies that the animals, when matured, respond to visual stimuli. They suggest that the “perceptual modality of a neo-cortical region is instructed to a significant extent by its extrinsic inputs” (von Melchner, Pallas and Sur 2000)8. The notion that the mode of perception is structured through the properties of the signal is reinforced by experiments in sensory substitution. For the last forty years, Bach-y-rita and colleagues have undertaken the design and testing of devices that aspire to replace vision for the

blind by means of tactile stimulations of the surface of the skin. Tactile Vision Substitution System devices typically consist of a video source mapped onto the surface of the skin by vibrotactile or electrotactile means. The back, abdomen, neck, forehead, fingertip and tongue have been used successfully. The blind subjects reported the ability to localize objects in three dimensional space, to identify shapes and to read bold, large, high-contrast text. To describe this phenomena as seeing risks an easy and direct comparison to normal ocular vision with which it would pale. The resolution is low and there are no colors, yet the astonishing fact is that objects can be apprehended in space remotely by the tactile sense and that perceptions obtained in this way obey the optical rules of parallax, looming, perspective and foreshortening, and overlap (Bach-y-rita 1972). A traditional view of the tactile would limit its utility to the apprehension of proximal stimuli. Here, here is evidence that it can do much more when augmented technologically. But, sensory substitution does not happen automatically and is not due solely to the presence of appropriately designed technological artifacts. Video images derived, for example, from a television program and applied to the skin would result only in the perception of vibrations on its surface rather than of objects located in space. Spatialization is entirely dependant on coupling changes in the video image to volitional movements that cause them. Bach-y-rita (1972) reports that a subject wearing a head mounted camera coupled to an abdominal output array will initially reach for an object at waist level, but with hours of experience in moving about an environment, will correctly perceive the object location relative to the head-mounted camera. A series of structured experiments by Lenay et al documents a direct correlation between the degree of spatialization and the degrees of freedom of movement. The device used for the experiments was simple photosensor switching on small vibrating motors in response to light. This is equivalent to reducing vision to one bit-mapped pixel. Even under these reduced conditions, Lenay, Canu and Villon(1997) report that with free movement in three dimensions comes a “spectacular ability to recognize forms…accompanied by an exteriorization of the percepts, which become objects located in space”. Sensory substitution research supports the notion that vision is precisely the relationship between certain transformations of neural patterns that ORegan and Noe (2001) term sensorimotor contingencies. These contingencies are available in sensory substitution systems and so the perception that arises is a kind of vision, even though it is transduced by cutaneous neural pathways. This research offers fundamental insights into the origins of spatial perception, the necessary and sufficient conditions that give rise to the perception of, not only spatially distributed objects and enclosures, but also through contradistinction to our awareness of the self. Two general classes of invariant patterns may be hypothesized, one corresponding to sensations that arise from internal sources indicating the condition of the body and changes in its disposition, the other, to changes that arise in respect to transitions in the internal sensations and the constancies that exist within the resulting flux. The first, we identify with the self and the second with the external world. This suggests that what we understand to be constancy in our physical surroundings is in fact internally generated (Maturana and Varela 1980). One implication of the view of that the both the self and the external world are the result of inferential processes rather than intrinsic properties is that it should be possible to modify what is experienced as ‘self’ or as ‘world’ by means of a specific couplings of organism and artifact. The body image varies with circumstance. When deprived of sensation from specific nerves the body image may contract. "The first night in space when I was drifting off to sleep," recalled one Apollo astronaut, "I suddenly realized that I had lost track of ... my arms and legs. For all my mind could tell, my limbs were not there. However, with a conscious command for an arm or leg to move, it instantly reappeared -- only to disappear again when I relaxed” (Barry and Phillips 2001). This effect is similar to that reported by users of sensory deprivation tanks9. Indeed, the support provided by the heavily salinated waters of the tank allows one to float effortlessly just as one does in microgravity. The gravitational vector by its resistance identifies the body to itself. In its absence, the identity of the body correspondingly changes. With a limb amputation one would expect the body image to contract, but it does not always do so. In phantom limb syndrome missing or amputated limbs still seem to carry sensation. Pain or itching occur and can be very difficult for the patient as the pain cannot be treated with medication

and the itch can not be scratched. Ramachandran (1998) suggests that areas of the brain formerly associated with the limb are ‘invaded’ by neurons from adjacent areas and the two areas can be activated together. Sensations propagating in the cortex from another area of the body to that formerly associated with the missing limb cause the retention of the limb within the body image. In developing novel treatments for patients with these conditions, Ramachandran notes that the body image of a person is very elastic. He has been able to effect the incorporation of inanimate objects into a person’s body image with simple techniques, such as the synchronous stroking of, for example, a hand and table or chair. His experiments suggest that “despite all its appearance of durability, [the body image] is an entirely transitory internal construct that can be profoundly modified with just a few simple tricks”. Transformation of the body image is implicated in perceptual adaptation, as well. Sekyama et al (2000) note that body image transformation precedes adaptation to reversed vision goggles. They report that in some cases, there may be a duplication of body images one normal and one reversed and that the subjects reference that which is most useful to them in the given task. Astronauts adjusting to microgravity report that their dreams change over the course of their adaptation to orbiting environments10. Initially, their dreams are much like those on earth, soon however, they begin to float although others in the dream do not as yet. Eventually, floating becomes the normative condition in the dreams. The reverse sequence often occurs on return to earth’s gravity. The dreaming parallels and perhaps is instrumental in the transition of body image from upright in the gravitational field to one that floats. The body image may be a key factor in our relationship to our environments providing not only definition but also characterizing capability. The incorporation of technological artifacts into the body image, their increasing transparency within the intentional framework, is a critical aspect of tool use. The ability of the user to merge with an automobile or other technological artifact is widely experienced. In order for this to occur, a close coupling and structured relationship must be developed between the user and the artifact so that the pattern of invariants is of the same class as those identified as the ‘self. But, the coupling with the environment is more complex than the incorporation of selected aspects of it into the body image. Krueger (2002) reviews work by Kirsh, Hutchins and Clark, noting that current characterizations of the mind tend to extend it into the environment and implicate the environment in cognitive processes. Kirsh’s (1995) analysis of the use of kitchen spaces shows that cognitive organization can be physically structured by the way in which the environment is ordered in use. Memory requirements for sequences or processes can be incorporated into the environment and thereby removed from one’s mental load. Hutchins’ (1995) study of aircraft navigation leads him to conclude that the proper level of analysis for the activity of landing a plane is the hybrid condition of the socio-technical system rather than in the individual or technology alone. Gregory (1981) uses the term ‘potential’ intelligence to refer to the intelligence that is embedded into the environment by previous cultural processes as opposed to ‘kinetic’ intelligence which is that which we exhibit in activity. There is a role that artifacts play in cognition (Norman 1991) in addition to their role in perceptual processes (Lenay, Canu and Villon 1997). Clark (1997) argues that the head is too limited a container for the mind and that the body and environment are integral to the processes to which we ascribe that concept. The ‘extended mind’ hypothesis is at odds with traditional computational models of human intelligence. The human mind is doubtless an able symbolic processor. Reading this paper should be proof enough of that. Yet to limit our understanding of the mind to the manipulation of symbols (however liberally defined) is to dismiss important aspects of its operation. As Maturana (1995) notes, “The mind is not in the head, the mind is in behavior”. Behavior implies both body and context. Humans and their environments are deeply intertwined. This paper reviewed the relationship between physics and physiology. Bodies subjected to altered physical conditions are vigorously adaptive. Bone porosity and muscle atrophy were cited as examples of this adaptation. Perception is altered as well, especially with respect to the fusion between sensory modalities, and for a sense of orientation. In this respect, micro-gravitational and virtual environments exhibit similar adaptations. Perceptual adaptations are made in response to many systematic changes in the visual field, from experimental glasses to new modes of language and perception. This has lead to a questioning of traditional models of the human that posit individual constancy and

independence from conditions. It has been suggested that the environment that seems objective is an inference drawn by the same process as we use to define ourselves. What this work suggests is that we are profoundly implicated in the environments that we perceive and create through relations of mutual definition that are both intimate and circular. The traditional view of an objective physical reality and atomistic individuals dismisses the critical role that the environment plays and in doing so undervalues the role of design. Design must be seen as a fundamental human activity not only in shaping the environment but in the way in which we construct it from our experiences. The human is not individual singular and cohesive as our introspection suggests, but adaptive, mutable, responsive and contingent. This new understanding is not meant to alter experience or to suggest that the experiences are incorrect. Perhaps, it is far more interesting to hold both the experience of cohesiveness and the evidence of contingency together simultaneously and to attempt to understand how one gives rise to the other. The Synthetic Senses Project This paper began by looking at the relationship between conceptual frameworks and the opportunities and limitations that they imply. Beer (2000) notes that a conceptual framework can have profound empirical consequences, “influencing the phenomena we choose to study, the questions we ask about these phenomena, the experiments we perform, and the ways in which we interpret the results of these experiments”. Beyond that, I believe, the test of a good theory is not only in its descriptive capacity but in the action it implies11. If the model of the human used for design is different, how does the revision open possibilities for design that did not exist previously? Is there something I can do that I could not before? One such project is the design of ‘synthetic senses’ for humans – the ability to perceive the world in new ways by enacting a structured relationship between humans and sensory prosthetics that give rise to percepts rather than mere representations. To generalize from ORegan and Noe(2001), sense modalities are dependant on and originate in sensory motor contingencies. Where the contingencies associated with vision are reproduced in the tactile domain, one is able to spatialize objects through the tactile sense. A systematic structuring of the relations between sensory patterns ultimately derived from manufactured sensor technologies and those given by locomotion, proprioception and the internal senses should result in exactly those contingencies leading to the externalization of a percept. Vision substitution for the blind is a noble effort. From the perspective of the blind person, a novel perception appears. These same principles can be applied to enable novel percepts for those who have a full complement of common human senses. Bach-y-rita (1972) notes that the brain mechanisms underlying sensory substitution systems should be similar to, or identical with, those that could be used for “sensory augmentation or supplementation”. Again, he suggests that to “constitute such systems it is only necessary to present environmental information from an artificial sensor in a form of energy that can be mediated by the receptors at the human-machine interface” and “ that it is possible to provide information from any device that captures and transforms signals from environmental sensors” (Bach-y-rita, e,t al, 2003). Lenay et al (1997) claim that technologies actually invent and transform the goals of human activity and that cognitive technologies can give rise to new modes of perception. However, no systematic efforts to develop synthetic senses have been uncovered. The Human Interface Laboratory in the School of Architecture at Rensselaer Polytechnic Institute has a group of researchers actively studying this possibility. We have begun to build devices to test the synthetic senses hypothesis. The objective of the initial studies is to give humans the direct perception of magnetic fields. Magnetic fields were chosen for the initial explorations not only because of the availability and economy of the sensors, but more importantly, because magnetic fields can be both immersive and object-like with respect to the scale of the body. Because of this, the motor field, the volitional movements that give rise to the percept, will differ for each of these scales. In the case of the earths magnetic field the issue is orientation as the body is small with respect to it. Relevant movements will be locomotive. For object-scale magnetic fields the movements will be primarily the exploratory/performatory movements of Gibson’s (1963) ‘active touch’ involving the arm, hand

and fingers. We are interested also in the way in which the intentional capacities of these acts differ and the impact that that may have on the acquisition of percepts. The development of novel perceptions tends to focus attention on the uniqueness of the new perceptual mode. But, perception of the mode itself tends to be a phenomenon of the laboratory – an object of directed attention or the result of controlled conditions. Perception as a lived experience is about objects, conditions or meanings apprehended through a multiplicity of senses simultaneously. The importance of the integration of sense modalities has already been underscored by the discussion of space and simulator sickness that results from the decoupling of sensory inputs. The cutaneous interface has been chosen for the initial explorations because our interest is in the ways in which the new sense modality can be incorporated into and calibrated in reference to existing sense modalities. Vision and audition are heavily relied upon in our culture. Tactile information over much of the skins surface has no presence in perceptual consciousness because of habituation due to our controlled environments. We propose that this ‘underutilized sensory bandwidth’ can be effectively exploited to bring additional dimensions of the environment into perception and to develop this new sensory capacity to augment a full complement of normative senses. In this way the sense modality can be developed as an additional property or dimension of the already occurring flux of sensations that is patterned into self and world. Our experience of solidity and constancy in the environment is a function of the invariants in the internal patterns that we experience. We have the impression that this is the totality of our world as it is the totality of our experience. We perceive but a minute fraction of the available spectra and there is much that is not yet available through scientific instrumentation12. Making available additional dimensions of perception will result in a richer experience of the world. It will also illuminate certain aspects of it of which we are normally not aware. The manufactured sensor responds in particular ways to the phenomena that it is made to measure. It is ordinarily quite easy to accept the fact that the output of the sensor and the reality that it is supposed to describe are not identical. When we fabricate a synthetic sense experience we do not have access to the phenomena itself but only to the output of the sensor. Considerable design effort is spent on translating the sensor output into a form that is available to the nervous system and considerable slippage occurs. With synthetic senses there is no question that we perceive but an analog of the world that we intend to experience. It is less clear to many that an identical process occurs in our biological senses. They too provide an output that is analogous to our environment but is not identical to it. What we perceive is not the world and is not reality, but is the output of a biological transducer that we believe co-varies with external phenomena. Our perception then is partial in several ways, first because it is an incomplete assay of what is available, second because it is a mere translation of that portion to which we are sensitive, and finally it is inherently inferential rather than a direct perception. Piaget suggests that during child development perception evolves from internal surface pattern to external spatial extension, but the origin of our understanding of the world as an external phenomenon is lost to memory. Notions of how it developed are based on observations of behavior. Experiments in synthetic senses and sensory substitution offer the possibility of examining this transition as both an experimental and experiential process. Warren McCollough (1965) posited an experimental epistemology13. The Synthetic Senses Project may be one approach to this. This paper started with the attempt to ‘redefine the human’ while, at its end the project appears to be an effort to, in a sense, redefine the environment. They are, in fact, different facets of one and the same project; the human and its environment are inextricably intertwined and this fact opens new possibilities for design. BIBLIOGRAPHY

Agre, P (1995) ‘The Soul Gained and Lost: Artificial Intelligence as a Philosophical Project’, in Stanford Electronic Humanities Review Barry, P and Phillips, T. (2001). ‘Mixed up in Space’. Science @ NASA, National Aeronautics and Space Administration, August 7, 2001 available at: http://science.nasa.gov/headlines/y2001/ast07aug_1.htm (10-20-2003) Beer, R. (2000). Dynamical Approaches to Cognitive Science. Trends in Cognitive Sciences 4(3):91-99. Brooks, R. A. (1991, April). Intelligence without Reason. Massachusetts Institute of Technology Artificial Intelligence Laboratory Memo 1293 (Http://www.ai.mit.edu/people/brooks/papers/AIM-1293.pdf) Bach-y-Rita, P. (1972). Brain mechanisms in sensory substitution. New York: Academic Press. Bach-y-Rita, P., Tyler, M., & Kaczmarek, K. (2003). Seeing with the Brain. International Journal of Human Computer Interaction15, 285-295. Clark, A. (1997). Being There: Putting Brain, Body and World Back Together Again, M.I.T. Press, Cambridge, MA Cooper (1976). A House in Space. New York: Holt Reinhart and Winston Gibson, J. J. (1963, March). The Implications of Active Touch [Mimeographed seminar notes - the ‘Purple Perils’]. Cornell University unpublished manuscripts available at: http://www.ksu.edu/psych/farris/gibson/files/acttouch.html Gregory, R.(1981) Mind in Science. London: Penguin Held, R. (1965). Plasticity in Sensory-Motor Systems. Scientific American, pp. 84-94. Fitts, R., Riley, D and Widrick, J. (2000) Physiology of a microgravity environment invited review: microgravity and skeletal muscle. Journal of Applied Physiology 89: 823-839 Hutchins, E. (1995). How a Cockpit remembers its Speeds, Cognitive Science, 19:265-288 Kirsh, D. (1995). The Intelligent Use of Space. Artificial Intelligence, 73, 31-68. (Http://icl-server.ucsd.edu/~kirsh/Articles/Space/AIJ1.html)

Krueger, T (1999) Intelligence and Autonomy. Convergence. The Research Journal in New Media Technologies, University of Luton Press, Luton, UK Vol. 5 no. 1, pp.91-101.

Krueger, T.(2000a). Architecture of Extreme Environments. in Armstrong, R. (ed) Space Architecture, Architectural Design, March 2000 Krueger, T. (2000b). On the Design of Organisms in Ascott, R. (ed) Consciousness Reframed: Art and Consciousness in the Post-Biological Era, Proceedings of the Third International Research Conference of the Center for Advanced Inquiry in the Interactive Arts, University of Wales College - Newport, UK Krueger, T. (2002). Des Interfaces aux medias non Symboliques, in Poissant ed. Des Esthetique Arts Mediatiques. Interfaces et Sensorialite, Preses de l’Universite du Quebec. Montreal, Canada Kohler, I.(1964). The Formation and Transformation of the Perceptual World. Psychological Issues Monograph 12, 3(4) New York: International Universities Press

Kohler, I. (1962). Experiment with goggles. Scientific American, 206, 62-72. Lenay, C., Canu, S., & Villon, P. (1997, August 25). Technology and Perception: The contribution of sensory substitution systems. 2nd International Conference on Cognitive Technology (CT ‘97). Aizu, Japan. Maturana, H. R., & Varela, F. J. (1980). Autopoiesis and cognition: the realization of the living. Boston studies in the philosophy of science. Dordrecht, Holland Boston: D. Reidel Pub. Co. McCollough, W. (1965) Embodiments of Mind. Cambridge, MA: MIT Press Norman, D. (1991). Cognitive Artifacts. In J. M. Caroll, Designing Interaction, Psychology at the Human Computer Interface. Cambridge: Cambridge University Press. O’Regan, J. K., & Noe, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5). Ramachandran, V. S., & Blakeslee, S. (1998). Phantoms in the Brain: probing the mysteries of the human mind. New York: William Morrow. Sacks, O. (1989). Seeing Voices. A journey into the world of the deaf. Berkeley, CA: University of California Press. Rock, I. (1984). Perception. New York: Scientific American Books. p.129 Sekiyama, K., Miyuchi, S., Imaruoka, T., Egusa, H., & Tashiro, T. (2000). Body image as a visuomotor transformation device revealed in adaptation to reversed vision. Nature, 407, 374-377. Simon, H. (1969). Sciences of the Artificial. Cambridge, MA: MIT Press Thompson, A. (1996). “An evolved circuit, intrinsic in silicon, entwined in physics” in Proceedings of the First International Conference on Evolvable Systems, Springer-Verlag, 1997, 390-405 Available at: http://www.cogs.susx.ac.uk/users/adrianth/ices96/paper.html (9/22/03) Thompson et al (1999). Explorations in Design Space: unconventional electronics design through artificial evolution. IEEE Transactions on Evolvutionary Computation vol 3 no 3 September 1999 Thompson, D’Arcy (1917) On Growth and Form. Cambridge: Cambridge University Press Unterman, S., Marquez, J. and Brown, E. (2002) ‘Muscle Changes in Weightlessness’ Space Biomedical Engineering and Life Support Course documents by Newman, D. at MIT available at: http://paperairplane.mit.edu/16.423J/Space/SBE/muscle/muscle_bckgrnd_ug.html (9/20/03) Von Melchner, L., Pallas, S. and Sur, M. (2000). Visual behavior mediated by retinal projections directed to the auditory pathway. Nature 404:871-876, 2000. Wann, J. P., & Ibrahim, S. F. (1992). Does limb proprioception drift? Experimental Brain Research, 91, 162-166. Notes

1 What is it like to experience the design project that you are currently working out? How does it appear, from where? How does it sound? What are its textures? Does it ever get dirty? What does it smell like? Is it humid? Is there a breeze? Is anyone with you? 2 This paper is written from a western, indeed American, perspective, yet it has, I believe, something to offer many contemporary designers. 3 A gate is eliminated by being connected instead to the input voltage or to ground. The gates in question must be allowed to interact with the circuit in order for the frequency discrimination task to be successfully completed. 4 Piezo material also changes shape under variations in voltage, although this aspect is of no immediate interest, it provides high frequency response in many audio speakers. 5 Dr David Wolf (MD, PhD and astronaut) in a presentation and discussion with University of Arkansas students at the NASA Johnson Space Center on 17 September 1999. 6 “It turns out that you carry with you your own body-oriented world, independent of anything else, in which up is over your head, down is below your feet, right is this way and left is that way; and you take this world around with you wherever you go.” Cooper (1976) 7 The hardware/software dichotomy is, of course, only that same Cartesian dualism wrought in silicon and code. 8 The experience that accompanies the presentation of visual stimuli for the re-wired ferrets is, unfortunately, unknown. 9 Personal experience of Seth Cluett, Seminar in Human Environment Interaction, Rensselaer Polytechnic Institute October 20, 2003 10 David Wolf (see note 5) 11 This comes as a truism from that social science phase of my remote past. I haven’t found the source, but acknowledge that it does not originate here. 12 Although we don’t know what these might be as yet, to assume otherwise simply takes more than the available arrogance. 13 This is just one of many insights offered by Ranulph Glanville, as principle advisor of my dissertation on the topic of sensory augmentation in progress at the Faculty of the Constructed Environment of the Royal Melbourne Institute of Technology. His influence on my understanding of these topics over several years has been profound and pervasive.