brain asymmetry and the processing of native, second, and...

Brain Asymmetry and the Processing ofNative, Second, and Artificial Languages

Chuansheng ChenDepartment of Psychology and Social Behavior

University of California, Irvine

UCSD Cognitive Sciences 200, 11/13/2006

An International CollaborativeResearch Team

Prof. Qi Dong, BNU Dr . Gui Xue, UCLA

Li u Li , Li Ti an, Xue Feng

Ji ang Ti ng, Mei Lei l ei ,

He Qi nghua, BNU

Guo Yi , Hust on U.

Zhao Li bo, U. I owa

Dr . Kewei Chen, ASU

Dr . Yapeng Wang, BNU

Dr . Xi nl i n Zhou, BNU

Dr . Hongchuan Zhang,UCSD

Pr of . Zhen Ji n, B 306

Pr of . Danl i ng Peng,BNU

Zhang Lei , Zeng Yawei Zhen Dong, Li Ke, B 306

Outline I. Left-hemisphere dominance in language processing

– Two likely exceptions: Chinese, second languages

II. Special features of Chinese language and the righthemisphere– Reading (Study 1) and tone processing (Study 2)

III. Right brain and second-lang. (English) processing– Working memory (Study 3) and language switching (Study 4)

IV. Brain asymmetry and artificial language learning:– Visual words and phonological learning (Studies 5-8)

V. Overall conclusions and implications for theunderstanding of neural basis of language learning– Specifically, language-specificity, neural compensation, and

individual differences.

I. Hemispheric Lateralization of Language

About 95% of the total (Western) populationshows left hemispheric specialization forlanguage:– Almost all right-handed people– About 1/3 of the left-handed people

Left dominance may have resulted from theright-handed gestural system ofcommunication that preceded the origin oflanguage.

Two likely exceptions

Chinese language, because of itsspecial features

Second language, because ofcompensation?

II. What is special about Chineselanguage?

Pictographic Origins Tonal Addressed phonology

商 代 甲 骨 文 上 的 象 形 文 字

House Crossing Woods Eye

Cup set Tongue Hand Elephant

鼠牛 虎 兔

龙蛇

马羊

猴鸡 狗 猪

Mouse Ox Tiger Rabbit

Dragon Snake Horse Ram

Monkey Chicken Dog Pig

Neural Substrates for Reading

Fiez & Petersen, 1998

Left fusiform (labeledas Visual Word Form Area(VWFA))

– Words– Pseudowords (e.g.,

Joki)

Right fusiform– Faces– Pictures of objects

Hemispheric specialization:Words on the left, faces/pictures on the right

Tark

iain

en e

t al.,

200

2

What about (pictographic) Chinese then?

Stress vs. Tonal Languages

Two phonological units:– Segmental units (vowels, consonants)– Suprasegmental units (pitch, tone, stress)

English is a stress language: Stress does notprovide much lexical information. Tomato:toMAto, but no TOmato, tomaTO. (with a fewexceptions such as, CONtent, conTENT).

In contrast

Chinese is a tonal language. For example, 20 characters are pronounced

“ma”, with 4-6 characters for each of the fourtones.

Thus, tone is essential to the lexicalprocessing.

“Tones, like consonants, are listed in thelexicon as unit phonemes.” (Packard, 1986)

Separate neural systems for segmentaland suprasegmental units? If separate,does the processing of Chinese tonesrely more on right hemisphere fortone/music processing?

Addressed vs. assembled phonology Alphabetic languages typically have

assembled phonology, but some (e.g.,Italian,called shallow orthographies, Paulesu et

al.) are easier to assemble than others (e.g.,English, called deep orthographies).

Some logographic languages (i.e., Korean)use assembled phonology.

Chinese mostly uses addressed phonology-------------------------------- Assembled phonology: One can read/sound it

out a word without understanding it. Addressed phonology: One can understand a

word without being able to sound it out.

Ability to read aloud words

lower grade level grade level beyond grade level

Chinese

American

Lee,

Utta

l, C

hen,

199

5

The addressed vs. assembled phonology, however,is not expected to vary by hemispheres .

So is there evidence of a specialrole of the right hemisphere inthe processing of Chinese?

Past Research on Chinese

Hemifield Experiments– Some found right hemispheric dominance

(Cheng & Yang, 1986, Brain Lang.; Tzeng et al., 1979,Nature)

– Some found left hemispheric dominance(Besner et al., 1982, Brit. J. Psych.)

– Some found bilaterality (Fang, 1997, J. Exp Psych;Leong et al., 1997, Brain Lang.)

– Gender differences (with the foveal splitting method,Hsiao & Shillcock, 2005)

– There is a need to localize possible hemispheric asymmetry

Past Research (Cont’d)

ERP and fMRI Experiments

– All 20 studies found left hemispheric dominance inthe frontal region

– But with bilaterality or rightward lateralization inthe occipital and occipito-temporal regions.

Direct comparisons between the twohemispheres are needed to pinpoint thelocation of Chinese-specific processing

Study 1: Reading Chinese Characters Question: Which is Chinese-specific area in

the visual cortex? Hypothesis: Bilaterality or rightward laterality

occurs at the primary visual cortex, but wholeword processing is left lateralized.

Rationale: The visual cortex is hierarchicallyorganized:– Primary visual cortext (BA 17 and 18)

visuospatial tasks visual features of words– Higher-level visual cortex (BA 19 and 37) object

processing whole word recognition– Chinese is no longer a pictographic language.

Xue, Dong, Chen, Jin, Chen, Zeng, Reiman, 2005, Cog. Brain Res.

Hierarchical organization ofvisual cortex

Source: Riesenhuber & Piggo, 1999

Tasks

a “black [hei(1)]” and “white [bai(2)]” for thesemantic task

b “ticket [piao(4)] “ and “jump [tiao(4)]” for thephonological task.

Semantic Task vs. Fixation

Phonological Task vs. Fixation

Voxel-wise comparison

Original Flipped

Subtraction

Brain Asymmetry: Semantic Tasks

Brain Asymmetry: Phonological Task

Summary of Study 1 Our hypothesis was confirmed: significant

leftward asymmetry in the fusiform region(like alphabetical languages) but bilaterality inthe primary visual cortex (unlike alphabeticallanguages) for Chinese processing.

The right hemisphere appears to be importantfor the initial visual processing of the complexspatial features of Chinese characters.

Study 2: Chinese Tone processing Question: Where in the brain does Chinese

tone processing (suprasegmental) vary fromvowel processing (segmental)?

Hypothesis: Right hemisphere, possibly areasin the inferior frontal gyrus

Rationale: Tone processing resembles musicprocessing, which is right lateralized.

Previous research showed mixed results,perhaps due to single task and lack of directcomparisons between the two hemispheres.

Liu, Peng, Ding, Jin, Zhang, Li, Chen., 2005, NeuroImage

We used an adaptation paradigm (i.e.,keeping the tone constant but varyingthe vowel, or keeping the vowelconstant but varying the tone) and twotasks (pinyin and Chinese characters)to study the neural overlap ordissociation between tones and vowels.

Vowels Tones

Clear leftward laterality, but…

Differences between tones and vowels

Crosshair marks right inferior frontal gyrus.

Conclusion of Study 2

Left dominance in both vowel and toneprocessing.

Neural dissociation in tones and vowels forChinese, suggesting special neural bases(especially right IFG)for tonal languageprocessing (also see Gandour et al. Wang et al.).

Summarizing Studies 1 and 2 andstudies by others In general leftward lateralization for Chinese

processing (both visual words and sounds). Chinese tones and initial visual processing

showed right-hemisphere involvement (IFGand BA 17 and 18)

Thus, in all there are three main differencesbetween neural bases of Chinese and Englishprocessing: Right primary visual cortex, rightinferior frontal gyrus, and left dorsal lateralfrontal region (see next slide)

Based on meta images From: Bolger et al. (2005). Cross-cultural effect on the brain revisited. Hum. Brain Mapp

Green circles: occipitotemporal boundary: All languages [VWFA]Blue circles: dorsal inferior frontal area: all languages [motor/speech]Yellow circles: ventral inferior frontal region (more lateral for Japanese): alllanguages [speech]Red circles: superior posterior temporal and inferior parietal region: English andKana, but not Chinese and Kanji [graphophoneme conversion]Pink circles: Dorsal lateral frontal region: Chinese (and Kanji) [addressedphonology, not assembled phonology]Brown circle: Right primary visual cortex and right IFG: Chinese

rOTBSum

mar

y da

ta

rIFG

III: Right Brain and Second-Language Processing

There is much evidence that the rightbrain is involved in second languageprocessing, especially at the earlystages (e.g., nonfluent bilinguals).

But why is it important? Compensation?

Study 3: Second languageprocessing Working memory task:

Xue, Dong, Jin, Chen, 2004, NeuroImage

Red: Chinese

Green: English

Conclusions: Study 3 Similar working memory systems for native

and nonfluent second languages Greater activation in both hemispheres, but

especially the right hemisphere for thenonfluent second language.

Suggesting compensation?– Maybe, but with fluency confounded with L1 and

L2, it is not clear whether it is compensation orlanguage differences (L2 in the right hemisphere).A training paradigm is needed.

Other cognitive functions such as thoseinvolved in inhibiting L1? (especially theprefrontal areas for executive functions?)

Study 4: Language switching

ER design. Naming drawings of objects. Conditions:

– Switching: ChineseEng; EngChinese– Non-switching: Chi Chi; Eng Eng.

Wang, Xue, Chen, Dong, Xue. in press, NeuroImage.

Brain Region BA Coordinates z p

x y z

Frontal _Mid_R BA10 36 59 5 3.47 0.000

Frontal _Mid_R BA46

39 50 9 3.57 0.000

Frontal _Mid_Orb_R BA46

45 52 0 3.01 0.001

Frontal _Mid_Orb_R BA11

3 43 -12 3.45 0.000

Cingulum_Ant_R BA32

6 50 17 3.19 0.001

Occipital_Sup_R BA18

18 -89 27 3.36 0.000

Precentral _R BA 6 39 2 50 3.97 0.000

Frontal_Sup_Medial_L

0 29 46 3.72 0.000

Parietal_Sup_L BA 2 -33 -47 60 3.12 0.001

supraMarginal _ L BA40

-62 -36 35 3.15 0.001

Angular_L BA39

-42 -56 39 3.09 0.001

Temporal_Mid_L BA21

-62 -29 1 3.71 0.000

Brain regions activated by forward switching (Chinese English) relative tobackward switching (English Chinese).

Brain Region BA Coordinates z p

x y z

Occipital_ Inf_ R BA19 33 -85 -3 4.17 0.000

Lingual_R BA17 3 -73 -4 3.46 0.000

Cerebellum_L BA37 -24 -48 -23 4.21 0.000

Cerebellum_L BA19 -30 -62 -20 3.24 0.001

Fusiform_L BA37 -36 -59 -10 3.27 0.001

Precentral_L BA6 -33 5 36 3.29 0.001

Brain regions activated by backward switching relative to forward switching.

Study 4: Conclusions

When switching between Chinese andEnglish, executive control areas areespecially important. Much right hemisphereaction is due to that reason. Perhaps thatexplains to some extent the role of righthemisphere for second language processing

This study also confirmed some language-specific areas: Right occipital for switching toChinese reading and left temporo-parietalregion for English (assembled phonology)

IV: Brain asymmetry and learning anartificial language

To study the effects of language experienceon the neural bases of language processing--neural plasticity, especially the righthemisphere.

More specifically, to examine whether the roleof the right hemisphere is compensatory

Finally, to explore whether the left-right shiftcould be neurofunctional predictors oflanguage learning efficiency

Logographic artificial language (LAL) :Korean Hangul characters (in different fonts)

Mismatched sounds (those different from Chinese and English)

Arbitrary meanings (Chinese translation, picture)

Training

Study 5: Fusiform and visual wordlearning

Xue, Chen, Jin, Dong, 2006a. J. Cog. Neuro. Xue, Chen, Jin, Dong, 2006b. NeuroImage

Pre-Training Results: Fusiform AreaAsymmetry Index:

Chinese: .55

Korean: .21

(-1 = 100% right lateralization;

0 = bilaterality

1 = 100% left lateralization)

Preliminary conclusions:

Evidence of compensation (both right and left hemisphere)

Second-language is more bilateral

However, two challenges to these preliminary conclusions…..

Challenge #1

If the right hemisphere plays only acompensatory role, post-training resultsshould no longer show activation in theright hemisphere (provided that trainingis effective behaviorally as well asneurally).

Training Effects: BehavioralResults

70%

75%

80%

85%

90%

95%

100%

Before After

Corr

ect

Rati

o

Chinese

Korean

400

450

500

550

600

650

700

750

800

Before After

Rea

ctio

n t

ime

(ms)

Chinese

Korean

Training effects at the neural level

It appears that the right hemisphere may not simply play acompensatory role when processing a new language.

Why bilateral then?

Challenge #2: Individual Results

It appears thatbilaterality wasonly at the grouplevel. Mostindividuals werenot bilateral.

Furthermore, individual differences in AIwere stable across training sessions.

r =. 778; p < .005

If right hemisphere hadplayed a compensatoryrole, training shouldhave resulted inleftward laterality.

Leftward lateralization facilitated learning

Pretraining AI Post-training AI

Post

-trai

ning

per

form

ance

Also: A caution against interpreting correlation betweenbrain activation and learning as evidence of brain plasticity.

Regression Analysis: Predicting post-trainingperformance

Note: RT = reaction time, AI = asymmetry index, R2 = .75

Predictors B (s.e.) β t p

Pre-training RT .12 (.12) .17 .97 .36

Pre-training AI -100.3 (22.4) -.79 -4.47 .002

Summary of Study 5

The right hemisphere perhaps plays acompensatory role (as does the left hemisphere)initially, but its sustained activation may havemore to do with individual differences in learninga new language.

Individual differences in the brain asymmetryduring the initial processing of second languagepredicted later learning. In this case, the use ofthe left fusiform seems to facilitate the learningof visual forms of words.

Studies 6 - 8: Does Sex Matter?

Much evidence that in languageprocessing males show more leftlaterality, whereas females show morebilaterality (e.g., Hsiao & Shillcock, 2006; Kansaku et

al., 2000; Shaywitz et al., 1995). Does that affect the learning of a new

language?

Chen, Xue, Dong, Jin, Li, Xue, Zhao, Guo, 2006. Neuropsychologia

-1

-0.8

-0.6

-0.4

-0.2

0

D6 D7 D8 D9 D10

!"##$%&'(")*+"$,,(+($)'-

.

/

Male

Female

Similarly, in phonological training:

He, Li, Xue, Chen, Dong, under reviewMei, Chen, Xue, et al. in preparation

Individual differences in left middle temporal gyrus predicted listening comprehension

Six months later

Dong, Mei, Xue, Chen, under review

V. Overall Conclusions andimplications The processing of Chinese shows

mainly left-hemispheric dominance, justas that of alphabetic languages. Tworegions of the right hemisphere play aspecial role in Chinese processing: theprimary visual cortex for visual wordsand the inferior frontal gyrus forChinese tones.

Overall Conclusions cont’d The sustained right-hemisphere involvement

in second-language processing may haveless to do with compensation, more to do withlanguage switching and with individualtendencies to use one side of the brain whenlearning a new language.

Native-language-tuned brain facilitates theacquisition of a new language. Because ofsex differences in native language tuning (leftlaterality for males, and bilaterality forfemales), brain asymmetry differentiallypredicted learning for males and females.

Thank you for yourcomments & suggestions