responses to nonverbal behaviour of dynamic virtual characters in high-functioning autism

12
ORIGINAL PAPER Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism Caroline Schwartz Gary Bente Astrid Gawronski Leonhard Schilbach Kai Vogeley Published online: 4 August 2009 Ó Springer Science+Business Media, LLC 2009 Abstract We investigated feelings of involvement evoked by nonverbal behaviour of dynamic virtual characters in 20 adults with high-functioning autism (HFA) and high IQ as well as 20 IQ-matched control subjects. The effects of diagnostic group showed that subjects with autism experi- enced less ‘‘contact’’ and ‘‘urge’’ to establish contact across conditions and less ‘‘interest’’ than controls in a condition with meaningful facial expressions. Moreover, the analyses within groups revealed that nonverbal behaviour had less influence on feelings in HFA subjects. In conclusion, dis- turbances of HFA subjects in experiencing involvement in social encounters with virtual characters displaying non- verbal behaviour do not extend to all kinds of feelings, suggesting different pathways in the ascription of involve- ment in social situations. Keywords Mentalizing High-functioning autism (HFA) Nonverbal behaviour Virtual characters Introduction Social and emotional peculiarities are crucial for the understanding of autism spectrum disorders (hereafter: ASD). The processing of gaze and other nonverbal facial cues has been of particular interest. There is a large body of research focusing on the effects of gaze behaviour in healthy subjects on variables related to person perception, such as the attractiveness of the interaction partner (e.g. Williams and Kleinke 1993) or personality ratings (e.g. Larsen and Shackelford 1996). For example, Mason et al. (2005) showed that people whose gaze addressed the viewer received higher likeability-ratings than persons who showed averted gaze. Facial expressions are another important nonverbal cue with an impact on person per- ception, e.g. a smile leads to more positive evaluations than neutral expression (Otta et al. 1994), whereas crying per- sons are judged to be less aggressive and evoke stronger feelings of sadness in the observer (Hendriks and Vinge- rhoets 2006). Such processing of nonverbal cues is held to occur mostly automatically and intuitively in healthy subjects (Lakin 2006). Previous studies on nonverbal behaviour processing in ASD focused on the investigation of basic constituents of social cognition such as gaze direction detection and face processing. For example, ASD subjects have been found to be impaired in spontaneous gaze following (Leekam et al. 1997). Swettenham et al. (2003) and Kyllia ¨inen and Hie- tanen (2004) showed that control children as well as chil- dren with ASD reacted faster to a target stimulus which had been indicated before by the gaze direction of another person. However, this so-called validity effect (Nation and Penny 2008) does not necessarily imply that persons with ASD process gaze behaviour as a social cue. Vlamings et al. (2005) showed that arrows and eye stimuli triggered attention in the same way in autistic persons whereas control subjects showed different reaction times for social cues. This absence of a preferential reaction to gaze in contrast to geometric cues has been found repeatedly (e.g. Senju et al. 2004; Ristic et al. 2005) and might be C. Schwartz (&) G. Bente Humanwissenschaftliche Fakulta ¨t, Department of Social Psychology, University of Cologne, Herbert-Lewin Street 2, 50931 Cologne, Germany e-mail: [email protected] A. Gawronski L. Schilbach K. Vogeley Department of Psychiatry, University of Cologne, Kerpener Street 62, 50924 Cologne, Germany 123 J Autism Dev Disord (2010) 40:100–111 DOI 10.1007/s10803-009-0843-z

Upload: caroline-schwartz

Post on 14-Jul-2016

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

ORIGINAL PAPER

Responses to Nonverbal Behaviour of Dynamic VirtualCharacters in High-Functioning Autism

Caroline Schwartz Æ Gary Bente Æ Astrid Gawronski ÆLeonhard Schilbach Æ Kai Vogeley

Published online: 4 August 2009

� Springer Science+Business Media, LLC 2009

Abstract We investigated feelings of involvement evoked

by nonverbal behaviour of dynamic virtual characters in 20

adults with high-functioning autism (HFA) and high IQ as

well as 20 IQ-matched control subjects. The effects of

diagnostic group showed that subjects with autism experi-

enced less ‘‘contact’’ and ‘‘urge’’ to establish contact across

conditions and less ‘‘interest’’ than controls in a condition

with meaningful facial expressions. Moreover, the analyses

within groups revealed that nonverbal behaviour had less

influence on feelings in HFA subjects. In conclusion, dis-

turbances of HFA subjects in experiencing involvement in

social encounters with virtual characters displaying non-

verbal behaviour do not extend to all kinds of feelings,

suggesting different pathways in the ascription of involve-

ment in social situations.

Keywords Mentalizing �High-functioning autism (HFA) � Nonverbal behaviour �Virtual characters

Introduction

Social and emotional peculiarities are crucial for the

understanding of autism spectrum disorders (hereafter:

ASD). The processing of gaze and other nonverbal facial

cues has been of particular interest. There is a large body of

research focusing on the effects of gaze behaviour in

healthy subjects on variables related to person perception,

such as the attractiveness of the interaction partner (e.g.

Williams and Kleinke 1993) or personality ratings (e.g.

Larsen and Shackelford 1996). For example, Mason et al.

(2005) showed that people whose gaze addressed the

viewer received higher likeability-ratings than persons who

showed averted gaze. Facial expressions are another

important nonverbal cue with an impact on person per-

ception, e.g. a smile leads to more positive evaluations than

neutral expression (Otta et al. 1994), whereas crying per-

sons are judged to be less aggressive and evoke stronger

feelings of sadness in the observer (Hendriks and Vinge-

rhoets 2006). Such processing of nonverbal cues is held

to occur mostly automatically and intuitively in healthy

subjects (Lakin 2006).

Previous studies on nonverbal behaviour processing in

ASD focused on the investigation of basic constituents of

social cognition such as gaze direction detection and face

processing. For example, ASD subjects have been found to

be impaired in spontaneous gaze following (Leekam et al.

1997). Swettenham et al. (2003) and Kylliainen and Hie-

tanen (2004) showed that control children as well as chil-

dren with ASD reacted faster to a target stimulus which had

been indicated before by the gaze direction of another

person. However, this so-called validity effect (Nation and

Penny 2008) does not necessarily imply that persons with

ASD process gaze behaviour as a social cue. Vlamings

et al. (2005) showed that arrows and eye stimuli triggered

attention in the same way in autistic persons whereas

control subjects showed different reaction times for social

cues. This absence of a preferential reaction to gaze in

contrast to geometric cues has been found repeatedly

(e.g. Senju et al. 2004; Ristic et al. 2005) and might be

C. Schwartz (&) � G. Bente

Humanwissenschaftliche Fakultat, Department of Social

Psychology, University of Cologne, Herbert-Lewin Street 2,

50931 Cologne, Germany

e-mail: [email protected]

A. Gawronski � L. Schilbach � K. Vogeley

Department of Psychiatry, University of Cologne,

Kerpener Street 62, 50924 Cologne, Germany

123

J Autism Dev Disord (2010) 40:100–111

DOI 10.1007/s10803-009-0843-z

Page 2: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

indicative of a less intuitive processing of social cues in

ASD. In contrast, gaze cues play a particularly salient role

in healthy subjects, suggestive of specialized brain systems

(Baron-Cohen 1995).

With respect to facial expressions as another constituent

of non-verbal behaviour, Braverman et al. (1989) found

deficits in matching facial expressions in autistic subjects.

Impaired processing of emotionally salient expressions in

others is reported by many authors (e.g. Loveland et al.

1995; Celani et al. 1999). However, Ozonoff et al. (1990)

did not find differences between autistic children and

controls matched on verbal age in tasks such as matching

emotional expressions to sounds. Other researchers used

dynamic instead of static face cues and did not find deficits

in autistic subjects in inferring mental states from the eyes

(Back et al. 2007) or in matching videotaped emotional

expressions with photographs (Gepner et al. 2001). The use

of different paradigms might account for those conflicting

results. Dynamic and thus more realistic stimuli correspond

more to what has been learned in everyday life and might

thus probe the deficits of autistic subjects in processing

socio-emotional information more effectively.

Since Baron-Cohen’s influential theory-of-mind (ToM)

hypothesis of autism (Baron-Cohen 1995; Bruning et al.

2005) much research was performed on the capacity to

attribute mental states to others. While disturbances in

solving ToM tasks are a well-established finding (e.g.

David et al. 2008), autistic persons have also been shown to

perform normally in spite of exhibiting severe social

problems in their everyday life (e.g. Klin et al. 2003).

Despite the ability to acquire skills of mental state attri-

bution related to inferential and reflective processing of

social rules, deficits in spontaneous, intuitive and reflexive

mentalizing abilities or ‘‘undermentalizing’’ (Frith 2004)

are regarded as an important aspect for the explanation of

social impairments in autism. Indeed, it has also been

suggested that automatic and reflexive as compared to

controlled and reflective modes of understanding other

minds (e.g. Lieberman 2007) may rely upon different

neuronal networks which could be differentially disturbed

in ASD (Kennedy and Courchesne 2008). Moreover,

Campbell et al. (2006) found gaze processing and the

attribution of mental states to be associated in healthy

children whereas children with ASD were impaired in both,

hinting to a common neural network for intuitive gaze

processing and mental state attribution which might be

impaired in ASD.

In social encounters not only other-directed mentalizing

is important, but also the ability of attributing mental states

to oneself. Some authors propose that one’s own mental

experiences are a precondition for the capacity to attribute

mental states to others (e.g. Harris 1992), although this is

currently under debate (Carruthers 2009). As suggested by

the so-called simulation theory, the ascription of mental

states to others is essentially based on internal simulations

of mental states based on the observation of others’

expressive behaviours (Goldman 2006). In this process, the

ability to adequately perceive, judge and categorize one’s

own emotions is as important as perceiving and processing

the relevant cues in others’ behaviour. With regard to

autism, it has been proposed that not only other-directed

mentalizing is impaired but also the reflection on one’s

own mental states. Hill et al. (2004) investigated the pro-

cessing of one’s own emotions in ASD. They used a self-

report measure and found that ASD subjects reported more

problems in identifying and describing feelings than their

biological relatives and control subjects.

To our knowledge, no study so far addressed own

emotions in ASD in response to social stimuli. In the

present study we therefore investigated feelings of

involvement which were operationalised by measuring the

degree to which the subjects felt bored, interested,

addressed, delighted, annoyed, relaxed and nervous when

confronted with nonverbal behaviour of virtual characters.

These mental states cover the basic dimensions of evalu-

ation (e.g. ‘‘delighted’’) and activation (e.g. ‘‘relaxed’’)

according to the terminology of Osgood et al. (1957) and

reflected how involved the subjects are when confronted

with the virtual characters. Moreover, as a second set of

dependent variables, we measured the amount of ‘‘contact

experienced’’ and the ‘‘urge to establish contact’’ with the

virtual character, thereby covering involvement on a more

action-based level. As stimuli we made use of short ani-

mated video sequences containing virtual characters sys-

tematically varying gaze direction and facial expressions.

As described above, gaze and facial expressions are crucial

nonverbal behavioural cues for social cognition and most

probably also play an important role in attributing own

feelings of involvement. We used virtual stimulus material

since virtual characters allow for full experimental control

and elicit social responses similar to those evoked by real

humans (Bente et al. 1999). In contrast to this, even trained

human actors have been found to show subtle variations in

their nonverbal behaviour which are not intended by the

experimenter and are possible confounders. As own feel-

ings of involvement in autism in reaction to nonverbal cues

have not been studied so far, we prioritized experimental

control over ecological validity and opted for virtual

stimulus material.

On the basis of literature suggesting other- and self-

directed undermentalizing in autism we expected signifi-

cantly less experiences of ‘‘involvement’’, ‘‘contact’’ and

‘‘urge’’ to establish contact in ASD subjects compared to

control persons. In addition, we expected no effect of gaze

direction on the dependent measures within the ASD

group, as gaze and direct gaze processing in particular have

J Autism Dev Disord (2010) 40:100–111 101

123

Page 3: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

been found to be impaired in autism. At the same time we

hypothesized a significant positive effect of direct gaze (as

opposed to averted gaze) in the control group in response to

the same stimuli. With respect to facial expression we

expected no effect on the dependent measures within the

HFA group, but a positive effect of meaningful facial

expressions (as opposed to arbitrary ones) on the dependent

measures in the control group.

Methods

Study Participants

The clinical group for this study consisted of 20 subjects

aged 20–53 (11 males, 9 females) with the diagnosis of

high-functioning autism (HFA) or Asperger syndrome

(AS). Since research has not provided clear differences

between HFA and AS so far we do not differentiate between

HFA and AS in this study (in concordance with Frith and de

Vignemont 2005; David et al. 2008). In the following, we

therefore use HFA as an umbrella term for both. We

recruited all subjects at the autism outpatient clinic at the

Department of Psychiatry of the University of Cologne.

Control subjects were recruited through on-campus adver-

tisement. Twenty control subjects matched for gender,

years of education and IQ without any neurological or

psychiatric history were included in this study.

The study was embedded in a series of examinations

(clinical interviews, neuropsychological testings, structural

MRI) that were performed in four different sessions. Autistic

traits were confirmed by clinical interviews according to

ICD-10 criteria by two independent physicians. Addition-

ally, all HFA subjects were screened with the Autism

Spectrum Quotient (AQ; Baron-Cohen et al. 2001a). As

expected, the HFA group scored significantly higher on the

AQ (41.3 ± 3.7, data are mean ± standard deviation)

compared to controls (14.2 ± 5.3; F 1, = 347.85, p \ .01).

As depression is a common comorbid condition (Stewart

et al. 2006), the self-rating instrument Beck Depression

Inventory was applied (BDI: Beck and Steer 1987; Hautz-

inger et al. 1995). Subjects with HFA had significantly

higher BDI scores (F = 14.04; p \ .01). Demographic and

psychopathological variables for both groups are listed in

Table 1.

Participants with HFA were on average older than

control participants (39.3 ± 9.2 compared to 34 ± 7.2,

p \ .05). Emotional functioning as investigated in our

study is not known to deteriorate with age. On the contrary,

a longer learning history in the HFA group might be an

advantage for acquired mentalizing abilities emphasizing

even more possible deficits that occur in spite of older age.

Moreover, no significant changes in the capacity to ascribe

mental states to oneself are likely to occur in the age span

between ± 34 and ± 39 years. We thus assumed that the

group difference in age would not significantly influence

our results.

In addition to the experimental task, the intelligence test

WAIS-R and the ‘‘Reading the mind in the eyes’’ test

(ToM-Eyes: Baron-Cohen et al. 2001b) to assess mental-

izing abilities were conducted. The order of tests and

experiments was randomized across participants. The

questionnaire was answered by the participants before the

first clinical interview.

HFA subjects showed a similar level of education (years

of education: 18 ± 3.28 years) as controls (20 ± 4.36

years; F = 2.615, n.s.). Accordingly, the HFA group

yielded verbal (125.3 ± 11.3), performance (123.2 ±

13.4) and total IQ scores (127.2 ± 12) that were not sig-

nificantly different from the control group (verb. IQ:

125.8 ± 8; perf. IQ: 127.7 ± 11.1; tot. IQ: 131.6 ± 8.3;

all F \ 2, all p [ 0.05, n.s.) as assessed with the Wechsler

Intelligence Scale for Adults (WAIS-R, German version

HAWIE-R: Tewes 1991). As expected, control subjects

reached significantly higher scores on the ToM-Eyes test

(16.2 ± 4.07, HFA: 18.7 ± 3.01, F = 4.88, p \ .05). IQ

and mentalizing results are listed in Table 1. Furthermore,

the two groups were matched for handedness (F = 0.49,

n.s.) as assessed with the Edinburgh Handedness Inventory

(EHI: Oldfield 1971).

Table 1 Demographic, psychopathological, IQ and mentalizing

results

HFA Control Statistics

M (SD) M (SD)

Age (years) 39.3 (9.2) 34 (7.2) F1 = 4.22*

Gender (female: male) 9:11 9:11 v2 = .40 (n.s.)

EHIa 81.1 (22.3) 82.4 (16.2) F1 = .49 (n.s.)

BDIa 11.2 (7) 4.8 (3.1) F1 = 14.04**

AQ2 41.3 (3.7) 14.2 (5.3) F1 = 347.85**

WAIS-Rb verbal IQb 125.3 (11.3) 125.8 (8) F1 = .03 (n.s.)

WAIS-Rb performance

IQb123.2 (13.4) 127.7 (11.1) F1 = 1.34

(n.s.)

WAIS-Rb IQ (total)b 127.2 (12) 131.6 (8.3) F1 = 1.87

(n.s.)

ToM-Eyesa 16.2 (4.07) 18.7 (3.01) F1 = 4.88*

M mean, SD standard deviation, n.s. not significant, EDI = EHIEdinburgh Handedness Inventory, BDI Beck Depression Inventar, AQAutism Spectrum Quotient, WAIS-R Wechsler Intelligence Scale for

Adults, ToM-Eyes Reading the Mind in the Eyes Test

* p \ .05, ** p \ .01a Raw scoreb Standardized score

102 J Autism Dev Disord (2010) 40:100–111

123

Page 4: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

Stimulus Material, Experimental Design and Procedure

Stimulus material consisted of animated video sequences

which had been designed using the software package

Poser 4.0 (Curious Lab�). The material has already been

used before and was extensively studied with respect to

their neural correlates in healthy control volunteers before

(Schilbach et al. 2006; Schilbach et al. 2008; Mojzisch

et al. 2006). These short video sequences depict virtual

characters with head and shoulders (Fig. 1; for example

stimuli see http://www.medizin.uni-koeln.de/kliniken/psy

chiatrie/Bildgebung/schilbach.htm). The virtual characters

appeared on screen and exhibited dynamic facial

expressions resembling real-life approach situations when

initiating social interaction. Sequences systematically

varied in the following respects (for further details see

Schilbach et al. 2006): (1) Facial expressions of the vir-

tual characters are perceived by healthy control persons

as socially relevant [SOC] in that they are indicative of

someone’s intention to establish interpersonal contact

while facial movements are perceived as arbitrary and

socially irrelevant [ARB], thus systematically varying the

degree with which the virtual character expressed his/her

intention to communicate with the addressee; (2) Virtual

characters shown in the experiment either gazed directly

at the study participant [ME] or looked aside towards an

invisible addressee situated at an angle of approximately

30� [OTHER] to the left or to the right, thus systemati-

cally manipulating the observer’s self-involvement. The

two factors (1) GAZE DIRECTION [ME vs. OTHER]

and (2) FACIAL EXPRESSION [SOC vs. ARB] consti-

tuted a two-factorial-design. Condition-specific dynamic

changes in facial expression were modeled according to

the Facial Action Coding System (FACS) (Ekman and

Friesen 1978). Animations were realized by interpolating

images between the neutral and condition-specific facial

expressions as well as body positions of the virtual

characters. Video sequences were generated simulating a

100 mm focal width camera view. In the video files the

virtual characters appeared with a light grey background.

The temporal order of each video clip adhered to a

standard pattern of 7.5 s. Each sequence began with the

entrance of a virtual character (‘‘walk in’’), followed by

positioning (‘‘turn’’) either towards the observer or

towards someone else who is out of view following a

rigid time course (Schilbach et al. 2006, 2008). Previous

studies employing the same stimulus material yielded

reliable effects in test persons with respect to judgment of

the degree of social interaction and were associated with

the recruitment of anterior medial prefrontal cortex as key

region of social cognition (Schilbach et al. 2006). In the

present study, we presented a subset of the original

stimulus material with four male and four female virtual

characters to every participant, expressing either a

socially meaningful facial expression (SOC, a smile alone

or a smile combined with eye-lid movement) or an

arbitrary facial expression (ARB, cheak or mouth move-

ment). Half of them oriented their gaze towards the

viewer and thus addressed the subject (ME), half of them

oriented their gaze to a third invisible person to the left or

right of the participant (OTHER) in a two-by-two-design.

The systematic permutation of gender (male versus

female), facial expression (SOC versus ARB) and gaze

direction (ME versus OTHER) resulted in eight animated

Fig. 1 Stimulus examples

taken from Mojzisch et al.

(2006). Time axis in

milliseconds. The virtual

character walks in, shows direct

(ME) or averted (OTHER) gaze,

then shows a meaningful (SOC)

or not meaningful (ARB) facial

expression, then turns and walks

off

J Autism Dev Disord (2010) 40:100–111 103

123

Page 5: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

sequences that were presented in a randomized order to

every participant.

All subjects received standardized verbal instructions.

Study participants were told to be part of a scene with two

virtual characters, one of which would appear on the screen

throughout the experiment (as illustrated in Fig. 1). The

other person on the right or left side could not be seen at

any time during the experiment from the participants’ point

of view, but was told to stand close to the participant. The

virtual character on the screen could therefore address the

study participant (ME) or the invisible other (OTHER).

Subjects were further instructed to watch eight sequences

with the virtual character and to answer the questionnaire

that appeared on the screen after each sequence. The

questionnaire was computer-based. The next sequence was

started via mouse click by the participant.

Data Analysis

As dependent variables (4-point Lickert scale) served the

following emotion items: ‘‘During the sequence I felt

annoyed, bored, interested, addressed, relaxed, nervous,

delighted’’. The points of the Lickert scale were: ‘‘strongly

disagree’’, ‘‘disagree’’, ‘‘agree’’ and ‘‘strongly agree’’.

Responses were coded so that higher values represent

agreement. The first step of data analysis was a factor

analysis (main component analysis with varimax rotation)

for the seven emotional items. As described in the results

section these items load on two separate factors, namely

‘‘interest’’ and ‘‘ease’’, which were used subsequently as

dependent variables in the first MANOVA. In addition,

participants were asked how strongly they experienced

‘‘contact’’ with the virtual character and how strong their

‘‘urge’’ was to establish contact with the person represented

by the virtual character (4-point scale). ‘‘Contact’’ experi-

enced and the ‘‘urge’’ to establish contact were used as

dependent variables in a second MANOVA. One between-

subjects factor (diagnostic group: HFA versus control

group) and three within-subjects factors (gaze: directed

versus averted, ME versus OTHER; facial expression:

socially meaningful versus arbitrary, SOC versus ARB;

gender of the virtual character: male versus female)

constituted the independent variables for both analysis,

leading to two 2 9 2 9 2 9 2 MANOVAs as key analysis

instruments. Age, depression and ToM measures were

considered as potentially influencing the dependent vari-

ables. Therefore, we calculated correlations of these mea-

sures with all dependent variables. As these correlations

failed significance (see Table 2), age, BDI depression

values and ToM-Eye scores were not included as covari-

ates in the MANOVAs.

Our hypotheses also focus on the effects of the inde-

pendent variables within the HFA versus control group.

Whereas the main effect of group in MANOVAs shed light

on differences in amount of feelings, the analyses within

groups can reveal whether effects are absent in HFA. We

therefore conducted Bonferroni-corrected pairwise com-

parisons for the factors ‘‘gaze’’ and ‘‘facial expression’’

within each group, even when the interaction of group and

‘‘gaze’’ or group and ‘‘facial expression’’ had not reached

significance.

Results

The factor analysis for the seven emotion items revealed

two components (Table 3). The first component can be

labelled ‘‘interest’’ whereas the second component covers

feelings of ‘‘ease’’. Based on these results from factor

analysis, separate values for the two factors ‘‘interest’’ and

‘‘ease’’ were calculated for each subject and served as

dependent measures in the following analyses.

Main Effects of Group (HFA Versus Control)

The first MANOVA did not reveal a significant effect of

group, neither on ‘‘interest’’ (F1,37 = .905, p = .348,

Eta2 = .024) nor on ‘‘ease’’ (F1,37 = .597, p = .444.

Eta2 = .016). The second MANOVA showed a decrease in

the HFA group both of ‘‘contact’’ (HFA: M = 1.974,

SD = .102; control group: M = 2.294, SD = .100;

F1,37 = 5.011, p = .031, Eta2 = .119) and of ‘‘urge’’ (HFA:

Table 2 Pearson correlations between dependent variables and age,

ToM-Eye Data and BDI

‘‘Interest’’ ‘‘Ease’’ ‘‘Contact’’ ‘‘Urge’’

Age (years) .185 (n.s.) .042 (n.s.) -.029 (n.s.) .100 (n.s.)

ToM-Eyes2 -.032 (n.s.) .202 (n.s.) .141 (n.s.) .092 (n.s.)

BDI -.283 (n.s.) -.131 (n.s.) -.112 (n.s.) -.193 (n.s.)

n.s. not significant, ToM-Eyes Reading the Mind in the Eyes Test,

BDI Beck Depression Inventar

Table 3 Rotated matrix of factors for feelings

Variance explained ‘‘Interest’’ ‘‘Ease’’

41.4% 22.0%

Interested .8 .3

Addressed .8 -.0

Bored -.7 .1

Delighted .7 .4

Relaxed .0 .9

Nervous .0 -.8

Annoyed -.3 -.6

104 J Autism Dev Disord (2010) 40:100–111

123

Page 6: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

M = 1.645, SD = .130; control group: M = 2.194, SD =

.126; F1,37 = 9.205, p = .004, Eta2 = .199).

Main Effects of Gaze, Facial Expression and Gender

of the Virtual Character

Both MANOVAs revealed significant main effects of gaze

and facial expression on all dependent measures, irre-

spective of diagnostic groups (see Tables 4, 5). Gender of

the virtual character did not have a significant effect on any

dependent measure (see Tables 5, 6).

Interactions

The first MANOVA showed a significant 2-way-interaction

between group and facial expression for ‘‘interest’’

(F = 10.101, p = .003, Eta2 = .214), but no significant

interaction between group and gaze (see Table 6).

The simple main effect of group within the condition

SOC turned out to be significant (F = 4.684, p = .037,

Eta2 = .112). In this condition, HFA showed significantly

lower levels of ‘‘interest’’ than control subjects (see also

Table 9).

There was no significant interaction between gaze

and facial expression. The second MANOVA revealed a

significant 2-way-interaction between group and facial

expression for ‘‘urge’’ to establish contact (F1,37 = 4.962,

p = .015, Eta2 = .150).

In addition, there was a significant 2-way-interaction

between gaze and facial expression on ‘‘contact’’. Simple

main effects showed that the effect of facial expression

was not reversed by the interaction. Meaningful facial

Table 4 Main effects first

MANOVA

M mean, SD standard deviation

Effect of group HFA: M (SD) Control: M (SD) F(1,37) p Eta2

‘‘Interest’’ -.10 (.67) -.10 (.71) .91 [.20 .02

‘‘Ease’’ .10 (.42) .09 (.62) .60 [.20 .02

Effect of gaze ME: M (SD) OTHER: M (SD) F(1,37) p Eta2

‘‘Interest’’ .29 (.10) -.26 (.11) 31.79 \.001 .46

‘‘Ease’’ -.13 (.11) .15 (.12) 8.29 \.05 .18

Effect of facial expression SOC: M (SD) ARB: M (SD) F(1,37) p Eta2

‘‘Interest’’ .42 (.10) -.40 (.10) 92.26 \.001 .71

‘‘Ease’’ .12 (.11) -.11 (.13) 5.74 \.05 .13

Effect of gender of virtual character Male: M (SD) Female: M (SD) F(1,37) p Eta2

‘‘Interest’’ .05 (.64) .06 (.72) 2.03 [.20 .05

‘‘Ease’’ -.05 (.55) -.06 (.70) 2.39 [.20 .06

Table 5 Main effects second

MANOVA

M mean, SD standard deviation

Effect of group HFA: M (SD) Control: M (SD) F(1,37) p Eta2

‘‘Contact’’ 1.97 (.10) 2.29 (.10) 5.01 \.05 .12

‘‘Urge’’ 1.65 (.13) 2.19 (.13) 9.21 \.05 .20

Effect of gaze ME: M (SD) OTHER: M (SD) F(1,37) p Eta2

‘‘Contact’’ 2.66 (.10) 1.61 (.08) 70.41 \.001 .67

‘‘Urge’’ 2.14 (.11) 1.70 (.10) 18.90 \.001 .34

Effect of facial expression SOC: M (SD) ARB: M (SD) F(1,37) p Eta2

‘‘Contact’’ 2.37 (.08) 1.89 (.09) 32.19 \.001 .47

‘‘Urge’’ 2.12 (.11) 1.71 (.10) 17.26 \.001 .32

Effect of gender of virtual character Male: M (SD) Female: M (SD) F(1,37) p Eta2

‘‘Contact’’ 2.14 (.52) 1.93 (.71) 008 [.20 .00

‘‘Urge’’ 2.13 (.53) 2.12 (.52) .368 [.20 .01

J Autism Dev Disord (2010) 40:100–111 105

123

Page 7: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

expressions led to higher contact feelings in the condition

OTHER as well (F = 3.270, p = .049, Eta2 = .150).

None of the 3-way-interactions between group, facial

expression and gaze reached significance (see Tables 6, 7).

Pairwise Comparisons of Gaze Within HFA and

Control Group

Planned pairwise comparisons within the HFA group

showed that the effect of gaze direction was significant for

the dependent measures ‘‘interest’’ (t = 4.065, p = \ .001),

‘‘contact’’ (t = 5.320, p = \ .001),) and ‘‘urge’’ (t = 2.724,

p = .010), but not for ‘‘ease’’ (t = 1.691, p = .099). In the

control group, gaze direction had a significant effect on all

dependent measures: ME led to an increase of ‘‘interest’’

(t = 3.904, p = \ .001), a decrease of ‘‘ease’’ (t = 2.386,

p = .022), an increase of ‘‘contact’’ (t = 6.578, p = \ .001)

and a decrease of ‘‘urge’’ (t = 3.437, p = .001) (see Table 8

for means and standard deviations). Thus, the gaze variation

had less influence on the experiences as empirically

addressed by the four factors of ‘‘interest’’, ‘‘ease’’, ‘‘con-

tact’’ and ‘‘urge’’, notably, ‘‘ease’’ was not influenced in

HFA.

Pairwise Comparisons of Facial Expression Within

HFA and Control Group

The pairwise comparisons in the HFA group revealed a

significant positive effect of SOC (as opposed to ARB)

only for ‘‘interest’’ (t = 4.472, p = \ .001) and ‘‘contact’’

(t = 3.479, p = .001), but neither for ‘‘ease’’ (t = 0.522,

p = .605) nor for ‘‘urge’’ (t = 1.121, p = .271). In the

control group SOC (versus ARB) significantly impacted

all dependent measures, namely ‘‘interest’’ (t = 9.125,

p = \ .001), ‘‘ease’’ (t = 2.902, p = .006), ‘‘contact’’

(t = 4.560, p = \ .001), and ‘‘urge’’ (t = 4.804,

p = \ .001) in a positive way (see Table 9 for means and

standard deviations).

Table 6 Interactions first

MANOVAGroup*Gaze F(1,37) p Eta2

‘‘Interest’’ .03 [.20 .00

‘‘Ease’’ .20 [.20 .01

Group*Facial expression F(1,37) p Eta2

‘‘Interest’’ 10.10 \.05 .21

‘‘Ease’’ 2.72 [.10 .07

Gaze*Facial expression F(1,37) p Eta2

‘‘Interest’’ 4.09 [.05 .10

‘‘Ease’’ .46 [.20 .01

Gaze*Facial expression*Group F(1,37) p Eta2

‘‘Interest’’ .18 [.20 .01

‘‘Ease’’ 1.16 [.20 .03

Table 7 Interactions second

MANOVAGroup*Gaze F(1,37) p Eta2

‘‘Contact’’ .25 [.20 .01

‘‘Urge’’ .14 [.20 .00

Group*Facial expression F(1,37) p Eta2

‘‘Contact’’ 1.00 [.20 .03

‘‘Urge’’ 7.74 \.05 .17

Gaze*Facial expression F(1,37) p Eta2

‘‘Contact’’ 8.84 \.05 .19

‘‘Urge’’ 1.85 [.05 .05

Group*Gaze*Facial expression F(1,37) p Eta2

‘‘Contact’’ .02 [.20 .00

‘‘Urge’’ .15 [.20 .00

106 J Autism Dev Disord (2010) 40:100–111

123

Page 8: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

Discussion

Our study shows that deficits in processing socio-emotional

information not only relate to disturbances in person per-

ception and the ability to ascribe mental states to others but

additionally include disturbances in experiencing involve-

ment when employing virtual characters display nonverbal

behaviour. This is indicated by a significantly lower gen-

eral experience of ‘‘contact’’ and ‘‘urge’’ to establish con-

tact in HFA as well as less ‘‘interest’’ in HFA subjects

when confronted with socially meaningful facial expres-

sions. Moreover, the analyses within each group showed

that both gaze direction and facial expression had less

impact on the experience of involvement in HFA than in

control subjects.

Effect of the Diagnostic Group: ‘‘Undermentalizing’’

in HFA?

The impairment of mentalizing, i.e. attributing mental

states to other persons (Frith 2004; David et al. 2008), and

the impairment of the ability to process one’s own emo-

tions (Hill et al. 2004) are well-known findings in autism.

On this basis, we hypothesized a reduced level of feelings

of involvement in the HFA compared to the control group

in a social encounter mediated by a virtual character.

Corroborating our initial hypothesis, we found that HFA

persons generally experience less ‘‘contact’’ and are less

inclined to establish contact (decrease of ‘‘urge’’ to estab-

lish contact), regardless of the nature of the stimulus.

Reduced engagement in social interactions is among the

core symptoms of autism. Our findings suggest that a lower

impulse to get into contact with others or to be socially

entrained by others is a key component that can be dem-

onstrated also in social encounters conveyed by virtual

characters. Other researchers have found that autistic

children react more positively to humanoid robots than to

humans (Dautenhahn and Werry 2004). Difficulties that are

already present during such a confrontation with virtual

characters might therefore be even more pronounced when

interacting with real persons. Other findings suggest that

individuals with ASD are more reluctant to interpret virtual

characters as real social partners and to attribute social

meaning to ambiguous cues, whereas typically developing

subjects intuitively do so (Abell et al. 2000; Klin 2000). As

a consequence, virtual characters might not appear ‘‘real’’

enough for subjects with HFA, and the differences we

found might be less pronounced when they are dealing with

real humans. However, our data do not support this inter-

pretation. Pairwise comparisons actually showed signifi-

cant effects of the stimulus variations in HFA, even though

the effects were broader in controls (see below).

Contrary to our expectations, we did not find a generally

lower level of ‘‘interest’’ and ‘‘ease’’ in the HFA group.

Previous findings suggest that dynamic material can facil-

itate processing of social cues for autistic individuals (Back

Table 8 Means (standard deviations) of all dependent measures with respect to the independent variable gaze direction

Control group ‘‘Interest’’ ‘‘Ease’’ ‘‘Contact’’ ‘‘Urge’’

ME .36 (.46)a -.07 (.74)a 2.86 (.46)a 2.44 (.73)a

OTHER -.17 (.57)b .25 (.66)b 1.73 (.57)b 1.95 (.67)b

HFA ‘‘Interest’’ ‘‘Ease’’ ‘‘Contact’’ ‘‘Urge’’

ME .16 (.74)a -.25 (.66)a 2.49 (.80)a 1.84 (.60)a

OTHER -.36 (.73)b .05 (.85)a 1.48 (.47)b 1.43 (.54)b

a For ME versus, b For OTHER indicates a significant difference for this dependent measure, a For both indicates a non-significant difference

Table 9 Means (standard deviations) of the dependent variables with respect to the independent variable facial expression

Control group ‘‘Interest’’ ‘‘Ease’’ ‘‘Contact’’ ‘‘Urge’’

SOC .64 (.55)a .28 (.65)a 2.56 (.44)a 2.53 (.77)a

ARB -.45 (.47)b -.10 (.72)b 2.03 (.46)b 1.86 (.63)b

HFA ‘‘Interest’’ ‘‘Ease’’ ‘‘Contact’’ ‘‘Urge’’

SOC .15 (.74)a -.09 (.73)a 2.16 (.54)a 1.69 (.52)a

ARB -.35 (.68)b -.10 (.82)a 1.80 (.62)b 1.58 (.62)a

a For SOC versus, b For ARB indicates a significant difference for this dependent measure, a For both indicates a non-significant difference

J Autism Dev Disord (2010) 40:100–111 107

123

Page 9: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

et al. 2007) which might have been the case in our study as

well. In addition, it might be relevant that we only included

adult individuals with high IQ values in our sample. It is

thus possible that our subjects have learned over lifetime to

adequately express and read out feelings of ‘‘interest’’ and

‘‘ease’’ when confronted with social stimuli even though

they might have been impaired in doing so during their

childhood. This is also corroborated by the observation in

interviews that the majority of HFA subjects has developed

idiosyncratic rules that allow them to ‘‘survive’’ in social

situations (e.g. look and smile at other persons). This issue

that HFA subjects might be able to learn and to acquire a

social skill–as opposed to the recruitment of an intuitive,

prereflexive, presumably inborn capacity–has already been

previously proposed with respect to ToM (Klin et al. 2003).

How can we explain that there are no general group dif-

ferences in feelings of ‘‘interest’’ and ‘‘ease’’ but at the same

time significantly lower ‘‘contact’’ and ‘‘urge’’ experiences

in HFA? An adequate use of words like the ones used as

items in this study (e.g. being ‘‘interested’’, ‘‘addressed’’,

‘‘delighted’’, ‘‘relaxed’’ etc.) can possibly be acquired as an

‘‘emotional vocabulary’’ without being necessarily experi-

enced emotionally to the same degree as by non-HFA sub-

jects. It is still unclear whether autistic persons suffer from a

lack of inner emotional experience on a lower level of

information processing or whether the problems arise on a

higher level, e.g. in reflections and in theorizing about ones

own inner states. In both cases, the acquisition of an emo-

tional vocabulary, e.g. via observation of other persons’

verbally expressed feelings or behaviour, might nevertheless

allow to adequately refer to such behavioural patterns usu-

ally indicating ‘‘common’’ feelings. In contrast to this, we

assume that experienced ‘‘contact’’ and ‘‘urge’’ to get into

contact are less verbalised by other persons in everyday life,

so that autistic subjects may have fewer opportunities to

learn their meaning. An alternative explanation is that

feelings of contact imply an action (that is related to actually

getting into contact with others) and thus bear consequences

for the relation with the interaction partner. To express those

feelings might therefore imply a higher threshold for HFA

subjects than to express the more ‘‘detached’’ feelings of

‘‘interest’’ and ‘‘ease’’.

However, when confronted with a meaningful facial

expression, HFA expressed less ‘‘interest’’ than controls.

Thus, the deficit in experiencing ‘‘interest’’ appears to be

specific to this condition. Given the high functional per-

formance of the group studied we assume that the subjects

were able to correctly identify the facial expressions and

that the differences we found are not due to cognitive, but

emotional ‘‘hypomentalizing’’ (Frith 2004). Why was there

less ‘‘interest’’ in HFA only in the condition of meaningful

facial expressions? A possible explanation might lie in the

employment of dynamic facial expressions which required

spontaneous reactions to changes in the mouth and eye

region, whereas the gaze variation only concerned the eye

region. Adequate reactions to gaze variations might

therefore be easier to acquire.

Generally speaking, the differences between HFA and

controls revealed by our study can be explained in two

ways. Either they represent experiential differences or

differences in the ability to report feelings (underreport-

ing). The relationship between the experience of an emo-

tion (qualia) and its explicit self-attribution or verbalisation

is discussed controversially in the field of philosophy (e.g.

Schwarz-Friesel 2007). In psychological research only the

outcome of emotional experience, i.e. the self-attribution,

can be measured. The fact that HFA subjects did not

exhibit generally lower levels of ‘‘interest’’ and ‘‘ease’’

than controls provides strong evidence for the assumption

that there is no bias towards underreporting and that our

findings represent differences in experience.

Effect of Gaze Direction Within Groups: No Effect

in HFA?

Contrary to our hypothesis, we found significant effects of

gaze in HFA subjects. Both HFA and control subjects

expressed higher feelings of ‘‘interest’’ and ‘‘contact’’ in

response to direct gaze (although—as revealed by the

MANOVA described above—the general level of ‘‘con-

tact’’ feelings was lower in HFA subjects). Different rea-

sons might account for this finding. First, it might be due to

the passive character that only required observation instead

of interactive engagement with the virtual character. A

passive observation is easier to handle than an interactive

situation and has decreased ecological validity (Boraston

and Blakemore 2007). In addition, our gaze variation was

not parametrically varied, but was clearly presented either

as directed or averted gaze. This might have facilitated the

detection and interpretation of this comparably easy social

cue for HFA, assuming that the interpretation of obvious

social cues can be learned by applying a ‘‘rational’’ or

‘‘theoretical’’ strategy, even if the intuitive processing is

impaired. More subtle and interactive gaze cues might be

needed to reveal different reactions of HFA subjects, e.g.

employing a systematic parametric variation of duration of

directed gaze (Kuzmanovic et al. 2009). Interestingly, and

in contrast to our hypothesis, we found negative feelings,

especially less experiences of ‘‘ease’’ in response to

directed gaze in control subjects whereas the gaze variation

did not influence feelings of ‘‘ease’’ in HFA subjects. The

negative effect of direct gaze in control subjects may be

explained by context factors. Most previous research has

focused on the influence of gaze cues on person perception

(e.g. Mason et al. 2005). So far little is known about the

feelings of involvement that are evoked in the passively

108 J Autism Dev Disord (2010) 40:100–111

123

Page 10: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

observing viewer. In our study, the encounter with the

virtual character was instantiated in a highly controlled

fashion so that control subjects might have perceived more

pressure when being directly gazed at which might have

led to feelings of unease. However, with respect to the

purpose of this study, the fact that HFA subjects’ level of

‘‘ease’’ did not depend on gaze direction is more interesting

and relevant. Baron-Cohen et al. (2001) showed that HFA

subjects are impaired in attributing emotions to a person’s

representation that is based on the depiction of the person’s

eyes. Our results are in accordance with the reduced ability

to ‘mindread’ other persons’ gaze behaviour. As suggested

by Campbell et al. (2006), a common neural network might

be implied in gaze processing and mental state attribution,

however, a recent study of our own group showed differ-

ential neural mechanisms for gaze detection and gaze

evaluation (Kuzmanovic et al. 2009). Impairments of the

gaze evaluation component in autism might explain diffi-

culties in attributing mental states to the self when con-

fronted with gaze behaviour of a virtual character. An

alternative explanation for the lack of an effect on feelings

of ‘‘ease’’ might be the higher baseline in depression in our

HFA sample compared to controls. As the comorbidity of

depression and anxiety is high (e.g. Alloy et al. 1990),

higher anxiety levels in our clinical sample are probable.

On the other hand, as discussed above, our findings do not

show a group difference in the level of ‘‘ease’’, but the

problems of subjects with HFA seem to be connected to the

fine-tuning of these feelings in response to nonverbal sig-

nals. Therefore we assume that the differences between

HFA subjects and control persons in their responses to the

gaze variations are not related to the affective state of

participants. This assumption is supported by the fact that

we did not find significant correlations between BDI scores

and the dependent measures.

Effect of Facial Expression Within Groups

As expected, facial expressions of the virtual characters

had different effects on the HFA versus the control group,

thus corroborating our second hypothesis. Socially mean-

ingful facial expressions had a positive significant effect on

all dependent variables in the control group. The effect was

less prominent and not universal in HFA subjects, namely

without impact on ‘‘urge’’ to establish contact nor on

feelings of ‘‘ease’’. It is not plausible that the impaired

recognition of the meaningful facial expression was the

cause as in that case there would have been no effect on

feelings in HFA at all. This is in line with evidence for

intact identification of facial expressions in HFA (Gepner

et al. 2001). Similar to gaze variation, facial expression

influenced feelings of ‘‘ease’’ in the control group but not

in HFA subjects. Following the simulation theory’s

approach (Goldman 2006), this hypomentalizing phenom-

enon with respect to the attribution of mental states to

oneself might be closely related to the impairment in the

attribution of mental states to others.

Interestingly, we did not find a global deficit in the sense

of hypomentalizing with respect to feelings of ‘‘interest’’,

neither on a general level (no significant main effect of

group) nor in response to gaze direction or facial expres-

sions. The latter finding hints to intact fine-tuning in

response to nonverbal signals.

This pattern was different for feelings of ‘‘ease’’: there

was also no difference in the general amount, but when it

came to fine-tuning feelings of ‘‘ease’’ to nonverbal cues,

HFA subjects failed. For their level of ‘‘ease’’ it did not

make a difference whether they were gazed at or what

facial expression the virtual character expressed. Feelings

of ‘‘ease’’ (related to the adjectives ‘‘nervous’’, ‘‘relaxed’’,

‘‘annoyed’’) express an inner state without a concrete

object relation whereas feelings of ‘‘interest’’ are always

directed towards an object (interested by something). The

latter might therefore be easier to adopt in the usage of

concrete stimuli. Concerning ‘‘ease’’, HFA might have

learned to express an adequate amount on a general level,

but the adjustment to stimuli is possibly harder to deal

with. Generally speaking, there might be two pathways of

ascribing mental states to oneself. One might be intuitive

and spontaneous and allow for flexible adaptation to dif-

ferent stimuli. The second pathway might depend on

learning of external symbols such as observed facial

expressions and the association of verbal terms to such

external cues. Assuming that the intuitive, prereflexive

component of mentalizing is impaired in HFA, our findings

suggest that feelings of ‘‘ease’’ and ‘‘contact’’ might be

harder to acquire using the second pathway than feelings of

‘‘interest’’. Of course, our findings do not provide a formal

proof for this interpretation. Interestingly, however, recent

neuroimaging data suggests altered functional organization

of the large-scale neural network involved in social and

emotional processing in autism, but no alteration of the

functional organization of the network involved in sus-

tained attention and goal-directed cognition (Kennedy and

Courchesne 2008). Conceivably, intuitive mentalizing

related to the experience of ‘‘contact’’ or the ‘‘urge to

establish contact’’ as compared to more reflective mental-

izing with respect to ‘‘ease’’ and ‘‘interest’’ as part of the

task in our study could differentially recruit these networks

and might explain why autistic individuals perform dif-

ferently in these two respects. Neuroscientific approaches

might, therefore, help to shed light on possibly independent

processes which could have resulted in the findings

observed in our study.

J Autism Dev Disord (2010) 40:100–111 109

123

Page 11: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

Limitations of the Study

A possible objection to the method of this paper is that we

are investigating deficits in the processing of feelings in a

way that presumes correct description of feelings. How-

ever, the fact that we found similar results in the HFA and

control group for some measures suggests that HFA sub-

jects actually can deal with the instrument. In addition,

higher BDI scores in the HFA group show that they are

able to use questionnaires to express feelings (see also Hill

et al. 2004). We can thus assume that our method yields

valid results.

Another critical point is the use of virtual stimulus

material. The advantage of high experimental control has

to be weighed against possibly reduced ecological validity

that has to be debated. Even though there is evidence that

virtual characters evoke ecologically valid social responses

(Bente et al. 1999), only a replication with human stimulus

material can show whether the effects of the variables we

focused on are alike.

Thirdly, both IQ and years of education were high in our

sample. Cognitive capacities above average might have

facilitated compensation of intuitive social cognitive skills

in subjects with HFA. The deficits we found might even be

more severe in persons with lower IQ.

Conclusion

Mentalizing deficits in autism spectrum disorders include

impairments in the ascription of feelings of involvement in

response to nonverbal behaviour of virtual characters.

Relative to controls HFA express less feelings and within

the HFA group certain effects are absent which are present

in control persons. However, this deficit is not universal.

Different pathways in mentalizing might explain this pat-

tern of results and future studies making use of functional

neuroimaging techniques that could elicit the underlying

neural mechanisms could help to investigate this in further

detail.

Acknowledgments This work was supported by a project grant of

the German ministry of Education and Research (BMBF ‘‘Social

Gaze: Phenomenology and neurobiology of dysfunctions in high-

functioning autism (HFA)) to Gary Bente and Kai Vogeley.

References

Abell, F., Happe, F., & Frith, U. (2000). Do triangles play tricks?

Attribution of mental states to animated shapes in normal and

abnormal development. Cognitive Development, 15(1), 1–16.

Alloy, L. B., Kelly, K. A., Mineka, S., & Clements, C. M. (1990).

Comorbidity of anxiety and depressive disorders: A help-

lessness-hopelessness perspective. In J. D. Maser & C. R.

Cloninger (Eds.), Comorbidity of mood and anxiety disorders(pp. 499–543). Washington: American Psychiatric.

Back, E., Ropar, D., & Mitchell, P. (2007). Do the eyes have it?

Inferring mental states from animated faces in autism. ChildDevelopment, 78(2), 397–411.

Baron-Cohen, S. (1995). Mindblindness. An essay on autism andtheory of mind. Boston: MIT Press.

Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley,

E. (2001a). The autism spectrum quotient (AQ): Evidence from

asperger syndrome/high functioning autism, males and females,

Scientists and Mathematicians. Journal of Autism and Develop-mental Disorders, 31(1), 5–17.

Baron-Cohen, S., Wheelwright, S., & Hill, J. (2001b). The ‘Reading the

mind in the eyes’ test revised version: A study with normal adults

and adults with asperger syndrome or high-functioning autism.

Journal of Child Psychology and Psychiatry, 42, 241–252.

Beck, A. T., & Steer, R. A. (1987). Beck depression inventory–Manual. San Antonio: The Psychological Association.

Bente, G., Petersen, A., Kramer, N. C., & Buschmann, J. U. (1999).

Virtuelle Realitat im Forschungseinsatz. Ein Wirkungsvergleich

videovermittelter und computersimulierter nonverbaler Kom-

munikation. Medienpsychologie Zeitschrift fur Individual- undMassenkommunikation, 2, 95–120.

Boraston, Z., & Blakemore, S. J. (2007). The application of eye-

tracking technology in the study of autism. Journal of Physiol-ogy, 581(3), 893–898.

Braverman, M., Fein, D., Lucci, D., & Waterhouse, L. (1989). Affect

comprehension in children with pervasive developmental disor-

ders. Journal of Autism and Developmental Disorders, 19(2),

301–315.

Bruning, N., Konrad, K., & Herpertz-Dahlmann, B. (2005). Bedeu-

tung und ergebnisse der theory of mind-forschung fur den

autismus und andere psychiatrische erkrankungen. Zeitschrift furKinder- und Jugendpsychiatrie und Psychotherapie, 33(2), 1–12.

Campbell, R., Lawrence, K., Mandy, W., Mitra, C., Jeyakuma, L., &

Skuse, D. (2006). Meanings in motion and faces: Developmental

associations between the processing of intention from geomet-

rical animations and gaze detection accuracy. Development andPsychopathology, 18(1), 99–118.

Carruthers, P. (2009). How we know our own minds: The relationship

between mindreading and metacognition. Behavioral and brainsciences, 32(2), 121–138.

Celani, G., Battacchi, M. W., & Arcidiacono, L. (1999). The

understanding of the emotional meaning of facial expressions

in people with autism. Journal of Autism and DevelopmentalDisorders, 29(1), 57–66.

Dautenhahn, K., & Werry, I. (2004). Towards interactive robots in

autism therapy: Background, motivation and challenges. Prag-matics and Cognition, 12(1), 1–35.

David, N., Gawronski, A., Santos, N. S., Huff, W., Lehnhardt, F. G.,

Newen, A., et al. (2008). Dissociation between key processes of

social cognition in autism: Impaired mentalizing but intact sense

of agency. Journal of Autism and Developmental Disorders,38(4), 593–605.

Ekman, P., & Friesen, W. (1978). Facial action coding system: Atechnique for the measurement of facial movement. Palo Alto:

Consulting Psychologists Press.

Frith, C. D. (2004). Schizophrenia and theory of mind. PsychologicalMedicine, 34, 385–389.

Frith, U., & de Vignemont, F. (2005). Egocentrism, allocentrism,

and Asperger syndrome. Consciousness and Cognition, 14(4),

719–738.

Gepner, B., Deruelle, C., & Grynfeltt, S. (2001). Motion and emotion:

A novel approach to the study of face processing by young

autistic children. Journal of Autism and Developmental Disor-ders, 31, 37–45.

110 J Autism Dev Disord (2010) 40:100–111

123

Page 12: Responses to Nonverbal Behaviour of Dynamic Virtual Characters in High-Functioning Autism

Goldman, A. I. (2006). Simulating minds: The philosophy, psychologyand neuroscience of mindreading. New York: Oxford University

Press.

Harris, P. (1992). From simulation to folk psychology: The case for

development. Mind and Language, 7, 120–144.

Hautzinger, M., Bailer, M., Worall, H., & Keller, F. (1995). Beck-depressions-inventar (BDI), Testhandbuch (2nd ed.). Bern: Hans

Huber.

Hendriks, M., & Vingerhoets, A. (2006). Social messages of crying

faces: Their influence on anticipated person perception, emotions

and behavioural responses. Cognition and Emotion, 20(6),

878–886.

Hill, E., Bethoz, S., & Frith, U. (2004). Brief report: Cognitive

processing of own emotions in individuals with autistic spectrum

disorder and in their relatives. Journal of Autism and Develop-mental Disorders, 34(2), 229–235.

Kennedy, D. P., & Courchesne, E. (2008). The instrinsic functional

organization of the brain is altered in autism. Neuroimage, 39,

1877–1885.

Klin, A. (2000). Attributing social meaning to ambiguous visual

stimuli in higher-functioning autism and asperger syndrome: The

social attribution task. Journal of Child Psychology andPsychiatry and Allied Disciplines, 41(7), 831–846.

Klin, A., Jones, W., Schultz, R., & Volkmar, R. T. (2003). The

enactive mind, or from actions to cognition: Lessons from

autism. Philosophical Transactions of the Royal Society ofLondon, Series B, Biological Sciences, 358(1430), 345–360.

Kuzmanovic, B., Georgescu, A. L., Eickhoff, S. B., Shah, N. J.,

Bente, G., Fink, G. R. et al. (2009). Duration matters:

Dissociating neural correlates of detection and evaluation of

social gaze. Neuroimage, 46(4), 1154–1163.

Kylliainen, A., & Hietanen, J. K. (2004). Attention orienting by

another’s gaze direction in children with autism. Journal ofChild Psychology and Psychiatry, 45(3), 435–444.

Lakin, J. L. (2006). Automatic cognitive processes and nonverbal

communication. In V. Manusov & M. L. Patterson (Eds.), TheSAGE Handbook of Nonverbal Communication (pp. 59–77).

Thousand Oaks, CA: Sage Publications.

Larsen, R. J., & Shackelford, T. K. (1996). Gaze avoidance:

Personality and social judgments of people who avoid direct

face-to-face contact. Personality and Individual Differences,21(6), 907–917.

Leekam, S., Baron-Cohen, S., Perrett, D., Milders, M., & Brown, S.

(1997). Eye-direction detection: a dissociation between geomet-

ric and joint attention skills in autism. British Journal ofDevelopmental Psychology, 15(1), 77–95.

Lieberman, M. D. (2007). Social cognitive neuroscience: A review of

core processes. Annual Review of Psychology, 58, 259–289.

Loveland, K. A., Tunali-Kotoski, B., Chen, R., Brelsford, K. A.,

Ortegon, J., & Pearson, D. A. (1995). Intermodal perception of

affect in persons with autism or Down syndrome. Developmentand Psychopathology, 9(3), 579–593.

Mason, M. F., Tatkow, E. P., & Macrae, C. N. (2005). The look of

love: Gaze shifts and person perception. Psychological Science,16(3), 236–239.

Mojzisch, A., Schilbach, L., Helmert, J. R., Pannasch, S., Velich-

kovsky, B. M., & Vogeley, K. (2006). The effects of self-

involvement on attention, arousal, and facial expression during

social interaction with virtual others: A psychophysiological

study. Social Neuroscience, 1, 184–195.

Nation, K., & Penny, S. (2008). Sensitivity to eye gaze in autism: Is it

normal? Is it automatic? Is it social? Development and Psycho-pathology, 20(1), 79–97.

Oldfield, R. C. (1971). The assessment and analysis of handedness:

The edinburgh inventory. Neuropsychologia, 9, 97–113.

Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). Themeasurement of meaning. Urbana: University of Illinois Press.

Otta, E., Lira, B. B. P., Delevati, N. M., Cesar, O. P., & Pires, C. S. G.(1994). The effect of smiling and head tilting on person

perception. Journal of Psychology, 128(3), 323–331.

Ozonoff, S., Pennington, B. F., & Rogers, S. J. (1990). Are there

emotion perception deficits in young autistic children? Journalof Child Psychology and Psychiatry, 31(3), 343–361.

Ristic, J., Mottron, L., Friesen, C. K., Iarocci, G., Burack, J. A., &

Kingstone, A. (2005). Eyes are special but not for everyone: The

case of autism. Cognitive Brain Research, 24(3), 715–718.

Schilbach, L., Ritzl, A., Kramer, N. C., Newen, A., Zilles, K., Fink, G.

R., et al. (2006). Being with virtual others: Neural correlates of

social interaction. Neuropsychologia, 44, 718–730.

Schilbach, L., Eickhoff, S., Mojzisch, A., & Vogeley, K. (2008).

What0s in a smile? Neural correlates of facial embodiment

during social interaction. Social Neuroscience, 3, 37–50.

Schwarz-Friesel, M. (2007). Sprache und Emotion. Tubingen:

Francke.

Senju, A., Tojo, Y., Dairoku, H., & Hasegawa, T. (2004). Reflexive

orienting in response to eye gaze and an arrow in children with

and without autism. Journal of Child Psychology and Psychiatry,45(3), 445–458.

Stewart, M. E., Barnard, L., Pearson, J., Hasan, R., & O’Brien, G.

(2006). Presentation of depression in autism and Asperger

syndrome: A review. Autism, 10(1), 103–116.

Swettenham, J., Condie, A., Campbell, R., Milne, E., & Coleman, M.

(2003). Does the perception of eyes trigger reflexive visual

orienting in autism? Philosophical Transactions of the RoyalSociety, Series B, Biological Sciences, 358(1430), 325–334.

Tewes, U. (1991). Hamburg-Wechsler Intelligenztest fur Erwachsene,Revision 1991. Bern: Huber.

Vlamings, P. H., Stauder, J. E., van Son, I. A., & Mottron, L. (2005).

Atypical visual orienting to gaze and arrow-cues in adults with

high functioning autism. Journal of Autism and DevelopmentalDisorders, 35(3), 267–277.

Williams, G. P., & Kleinke, C. L. (1993). Effects of mutual gaze and

touch on attraction, mood, and cardiovascular reactivity. Journalof Research in Personality, 27(2), 170–183.

J Autism Dev Disord (2010) 40:100–111 111

123