brenda rapp department of cognitive science johns hopkins...

The literate mind and brain

Brenda Rapp

Department of Cognitive Science Johns Hopkins University

1

Spoken language: A human capacity with deep genetic roots that allows us to describe and explain the world

Written language:

– A relatively recent human inventions – Allows us to preserve ideas independently of the speaker,

across time and space • Vital for the accumulation of knowledge and science

Abraham Lincoln: • “Writing, the art of communicating thoughts to the mind

through the eye, is the great invention of the world...enabling us to converse with the dead, the absent, and the unborn, at all distances of time and space.” Speeches And Letters Of Abraham Lincoln, 1832 1865

2

Written language

Language-use survey (on-line, Mechanical Turk) • 3 age groups: 18-25, 26-35, 36+ • 4 language types:

• Spoken language/Non-electronic • Listening or talking in person

• Spoken language/Electronic • phone, TV, radio, movie, online video, podcast, audio book, Skyp)e

• Written language/Non-electronic • Reading or writing on paper : book, magazine, letter, notes, docs

• Written language/Electronic • Reading or typing in electronic media (text messaging, e-mail,

websites, online chatting, word processing, blogging, apps, computer programs, e-Readers)

3

Written language in the age of e-communication?

0%

10%

20%

30%

40%

18-25(N=152)

26-35(N=125)

36+(N=124)

Written Electronic Spoken Non-ElectronicSpoken Electronic Written Non-Electronic

Written language in the age of e-communication?

Conclusions: Written language is increasingly vital in everyday life

– Among young, tech-savvy individuals time spent reading and writing in electronic media can be greater than time spent with:

– Face-to-face spoken language – Spoken language in electronic media (telephone,

TV, radio, etc.)

Overview

1. Written language in the expert adult system: Neural evidence

2. Learning written language: Cognitive evidence

3. Learning and relearning written language: Neural approaches

7

Overview

1. Written language in the expert adult system: Neural evidence

2. Learning written language: Cognitive evidence

3. Learning and relearning written language: Neural approaches

8

•Spoken language, object recognition, navigation, etc. are evolutionarily old skills that have genetic basis

•Written language is an evolutionarily recent human invention (3000 AD)

•Unlike evolutionarily old skills, written language is explicitly taught and learned

• How does the human brain incorporate skills like reading and writing that are not specifically dictated by the genetic blueprint?

Written language

9

•How is the study of the neural bases of written language in the adult relevant to understanding written language learning in children?

The “expert” adult brain represents the end-state/goal for the child learner; understanding the end-state can shape teaching

Written language in the literate adult

10

Meta-analysis of neuroimaging studies of spelling Purcell, Turkeltaub, Eden & Rapp, 2011

Neuroimaging studies: During spelling (and reading) the brain recruits areas traditionally associated with various basic cognitive functions, revealing the multiple components of written language processing.

visual object processing

attention/spatial spoken language

Question: In the adult system, are the written language processes dependent or independent of these older, more basic skills?

Predictions: Neuroimaging

• If, in the adult system, written language is parasitic on evolutionarily older processes such as spoken language and visual object recognition, then: Written language should activate precisely the

same areas as these more basic processes.

12

Although learning to read and spell certainly build on visual networks, fMRI reveals that specialized orthographic areas develop.

Rapp & Lipka, 2011

faces

houses Reading

Spelling

Predictions: Cognitive Neuropsychology

• If, in the adult system, written language is parasitic on evolutionarily older processes such as spoken language and visual object recognition, then: Damage to the spoken language system should

necessarily be accompanied by written language deficits Damage to the written language system should

co-occur with difficulties in visual object recognition Damage to orthographic working memory should

co-occur with visuo-spatial working memory deficits

14

DPT

• 36 year-old, right-handed male

• Law degree, corporate tax attorney

• Resection for oligodendroglioma in left fusiform

15

DSN

• 67 year-old, right-handed female

• College degree, retired editor • Resection for meningioma in

left fusiform

Purcell, Shea & Rapp 2014)

16

DPT DSN

Spelling: Words X X

Reading: Words X X

Reading: Word comprehension (synonym judgment)

X X

Spoken: Word comprehension (synonym judgment)

normal normal

Visual object/face processing normal normal

-54 -52 -41 -32

Orthographic Working Memory Deficit Lesions (n=9) • Centered on the left intra-parietal sulcus • Traditionally associated with working

memory/attention

Left

• Is orthographic working memory idependent from visuo-spatial working memory?

Evaluated Visual-Spatial WM: • Corsi Blocks Test

• span = 5 is within normal limits

18

• The absence of a spatial WM deficit is consistent with domain-specific orthographic WM processing in left IPS region

Results: span = 4.8 (range 4-5); within normal limits

Is written language parasitic on evolutionarily older processes such as spoken language and object recognition?

• Written language certainly depends on spoken language, visual object processes and the working memory system during development

• However, in the competent, literate adult there is considerable independence of orthographic processing

• Neuronal recycling or neuronal repurposing occurs where areas designed for other cognitive functions are used for specialized orthographic processing/representation

19

Overview

1. Written language in the expert adult system: Neural evidence

2. Learning written language: Cognitive evidence

3. Learning and relearning written language: Neural approaches

20

Letter perception and learning (Wiley, Wilson and Rapp; under review)

1. Does expertise affect letter perception? • Compare naïve and expert observers

2. Are different alphabets perceived differently? • Compare visual feature rankings for Roman and Arabic

letters

21

22

• Two groups of participants: • Experts: proficient readers

of Arabic (n = 24) • Naïve: no prior exposure

to Arabic (n = 24) • Participants judged if two

shapes were physically identical

• Examined RTs and accuracy

Linear mixed effects modeling (LMEM)

24

1. Group: expert/naïve

2. Knowledge-based factors: phonological similarity, letter identity, motoric similarity, alphabetic order

3. Visual features : For each pair of letters, their similarity was calculated for each of 15 different visual features

Does expertise affect perception?

25

Knowledge-based factors

• Significant effects of shared identity and motor stroke similarity for the expert viewers only

Visual feature rankings

Are alphabets perceived differently? Within and across alphabet visual feature rankings

26

Learning affects letter perception The findings reveal a visual system that, with learning, dynamically adjusts its weighting of visual features

1. “Task-irrelevant” knowledge affects perceptual judgments • Knowledge-based factors influence perception (motoric strokes, letter

identity, etc.) 2. Expertise affects visual feature processing

• Naïve and expert observers weigh visual features differently and with experience, complex visual features are recognized more efficiently

3. Different alphabets are perceived differently • Even expert observers learn to view different alphabets differently

27

Cognitively Optimized/individualized of learning of first and multiple written languages/scripts

• Develop techniques for individualized evaluation of individual perceptual “profiles” that serves as basis for targeted teaching and interventions that will efficiently move learners from being naïve to expert viewers

28

Overview

1. Written language in the expert adult system: Neural evidence

2. Learning written language: Cognitive evidence

3. Learning and relearning written language: Neural approaches

29

Measuring neural changes in real-time orthographic learning (Schubert, Purcell & Rapp, in prep)

• While in the scanner, participants are taught new spellings; items are presented for 7 learning trials while brain responses are measured

• Results: Orthographic learning involves different types of changes in different brain regions: in cortical areas involved in long-term memory storage (fusiform g.) vs. areas involved in early learning stages (hippocampus)

Fusiform gyrus

Hippocampus

Measuring orthographic learning: Changes in resting state networks

• Resting-state fMRI records brain response while participants are ‘at rest” in a relaxed, awake state

• Advantages: for populations that have difficulty performing tasks in the scanner (children and individuals with disabilities)

• Can evaluate learning-related changes

31

Network-specific regions of interest (ROIs)

32

Default-Mode Network (DMN)

From Laird et al. (2009)

Reading Network (RdN)

From Turkeltaub et al. (2002)

Network Coherence: Correlation of RS-fMRI time series for ROIS within and across networks

• How does network coherence change with (re)learning?

0.000.050.100.150.200.250.300.350.40

RdN DMN

}

Reading network ROIs DMN network ROIs

Network Coherence: Correlation of RS-fMRI time series for within and across network ROIs Neuro-typical group

* *

Within vs. across network coherence values differences for: - Neurotypical individuals - Adults with acquired dysgraphia (pre-treatment)

Treatment-related normalization of within vs. across network coherence for adults with acquired dysgraphia:

pre vs. post treatment

Conclusions We are well-positioned to bridge to the teaching and remediation of reading and spelling in first and multi-language learners • Strong theoretical foundations • Neuroimaging methods • Analytical techniques Singapore is remarkably well-positioned with its multi-lingual population and education system

36

37