the multimodal turn ferrarelli m 2013 ispams

THE MULTIMODAL TURNTHE MULTIMODAL TURN

How Technology

is

Changing

the

Way

We Read, Write

& Think

ISP “Pbro. Antonio Sáenz”Primeras Jornadas de Lenguas

Prof. Lic. Mariana FerrarelliJune 2013

Overview•Multimodality: Mode

/

Affordances

/ Technological

Convergence

•Digital & Oral Culture•Sample

text

•Multiliteracies•Political

Implications

MultimodalityMultimodality

Social Semiotics

Signs

Context

Meaning made up of different modes

MODESocially shaped and culturally given semiotic resources for making meaning

WordsImageColour

MultimodalityMultimodality

Words nameImages showColours highlight

(Kress, 2003)

Combination

of

a variety

of

modes

or

languages

Perceptual experience

MultimodalityMultimodality

AFFORDANCES

Potentials for making meaning (Kress, 2010)

Multimodality Multimediality

Deals

with

how information

is

presented

Text

Book/PageImage

Photography

Moving

Image

Cinema/ TVSound

Radio/CD

MultimodalityMultimodality

TechnologicalTechnological ConvergenceConvergence

All

modes

can now

be conveyed

through only

one

medium:

the

screen

TechnologicalTechnological ConvergenceConvergenceWatch

TV / movie

Read

the

newspaperListen to

the

radio/music

Take

picturesRecord videosSend

e-mails

Facebook

& TwitterSearch

the

web

Diary

-

alarm

clock

–

calculator –

voice

recorder

Take

notes

Oral / Oral / PrintPrint / Digital Culture/ Digital Culture

Multimodality

is

not

new.

Human communication

is

(and

has always been) multimodal.

Oral Culture

ORALITY

WordsGestures

& facial expressions

Eye

contactProxemicsKinesicsIntonationVolume

Oral / Oral / PrintPrint / Digital Culture/ Digital Culture

Oral CultureWORDS

one

way

of

creating

meaning

(among

others)

Oral CultureWORDS

one

way

of

creating

meaning

(among

others)

Oral / Oral / PrintPrint / Digital Culture/ Digital Culture

Gestures & facial expressionsEye contactProxemicsKinesicsIntonationVolume

Print

Cultureone

way

of

creating

meaning

WORDS

Oral / Oral / PrintPrint / Digital Culture/ Digital Culture

WordsSentencesParagraphsPagesBooks

Oral / Oral / PrintPrint / Digital Culture/ Digital Culture

Digital CultureWORDS

one

way

of

creating

meaning

(among

others)

Digital CultureWORDS

one

way

of

creating

meaning

(among

others)

ColourImageAnimationSoundMusicMoving imageLayout

Oral / Oral / PrintPrint / Digital Culture/ Digital Culture

Reading

& Writing

No longer

associated

with

mechanical literacy

(pencil

& paper)

Digital CultureDigital Culture

Keyboard + Mouse + Screen(Cassany, 2002)

Thinking

New

cognitive

skillsThe

ability

to

search

for

& deal

with

huge

loads

of

information

efficiently. Select

DiscernConnect

Create

Digital CultureDigital Culture

Flight PathsA networked novel by Kate Pullinger, Chris Joseph

and participants.

Chapter 5: Paths Crossing.http://www.flightpaths.net/stories/pathscrossing.html

ImageSoundText

IMAGE:What

does

red suggest?

What

is

the

effect

of

the

blurred image

of

the

car?

What

does

the

button

in Arabic

say about

the

story/ chatacters/ setting?

How do all

these

elements contribute

to

the

creation

of

a

mysterious

atmosphere?

SOUND:What

does

the

eerie

music imply?

Where

does

the

story

take

place? What

does

the

sound

of

birds

say

about

this?How is

‘silence’

represented

in the

story?

TEXT:Who

is

‘he’? Does

this

ambiguity

build

up expectation?Why

does

the

narrator

feel

surprised

when

‘he’

speaks

‘perfect

English’?Is

there

any

repetition

of

words? Any

parallel

structures? (‘waited

for people’, waited

for

sirens’)

What

is

the

effect

of

alliteration/ consonance/ similes?Are there

any

sensory

images?

Multimodal TextsAndy Campbell:Inside: A Journal of Dreams (2001)Nightingale’s Playground (2010)http://www.dreamingmethods.com/portfolio.html

Chris Joseph: Bully for You (2008)Tube Lines (2013)http://www.chrisjoseph.org/

http://narrativa-postparentetica.blogspot.com.ar/

MultiliteraciesMultiliteracies

Social Semiotics

Multimodality

Education

Multiliteracies

Abilities

required

to

read, write, respond

to and

produce multimodal texts.

A multimodal approach to education

Draws on a wider variety of textsMoves the focus from a language to other forms of communicationAllows for diversity & differentiationRg & Wg increasingly screen-based Author as fluid and transitory roleAFFORDANCES: potentialities & limitations

Political Implications

Multimodal approach

We challenge the dominant model of literacy

written language

We allow for differentiation & diversity

POLITICAL CHOICE

SourcesSources andand CreditsCreditsGee, J.P. (2003), What Video Games Have to Teach Us about Learning and Literacy,

Palgrave Macmillan, New York.Gee, J.P. (2005) Why Video Games are Good for Your Soul: Pleasure and Learning,

Common Ground, Melbourne.Cope, B. and Kalantzis, M. (2004), ‘Text-made text’, E-Learning 1, 198–282.Jewitt, C. (2009) The Routledge

Handbook of Multimodal Analysis, Routledge, London.

Kress, G. (1990) Linguistic Process and Sociocultural

Change, Oxford University Press,

Oxford, England.Kress, G.(2003) Literacy in the New Media Age, Routledge, London.Kress, G. (2010) Multimodality: A Social Semiotic Approach to Contemporary

Communication, Routledge, London.Kress, G. and van Leeuwen, T.(1996) Reading Images: The Grammar of Visual Design,

Routledge, London.Lauer, C. (2009) Contending with Terms: “Multimodal”

and “Multimedia”

in the

Academic and Public Spheres, Computers and Composition

26 (2009) 225–239

Mavers, D. (2011) Children’s Drawing and Writing: The Remarkable in the Unremarkable, New York, Routledge.

New London Group.:(1996) ‘A pedagogy of multiliteracies: Designing social futures’, Harvard Educational Review

66, 60–92.

Oliver, M. (2005)The problem with affordance’The

E-Learning Journal 2 (4): 402-

413 Public Schools NSW (2012) Using digital and multimodal texts K-6, Department of

Education and Communities. Available at: www.tale.edu.au/. Retrieved: March 2013.

Siege, M. (2006) Rereading the Signs: Multimodal Transformations

in the Field of

Literacy Education; Language Arts, Vol. 84, No. 1, September 2006.Snyder, I. (2004) Alfabetismos

digitales. Comunicación, innovación y educación en la

era electrónica, Colección Aulae, número 2. Ediciones Aljibe, Enseñanza Abierta de Andalucía y Consorcio Fernando de los Ríos.

Stein, P. (2008) Multimodal Pedagogies in Diverse Classrooms: Representation, Rights and Resources, Routledge, London.

Unsworth

, L. (2001) Teaching multiliteracies

across the curriculum, USA, Open

University Press.http://www.dreamingmethods.com/portfolio.html; allposters.com;

howfreeisfreedom.wordpress.com; http://www.chrisjoseph.org/; www.freefoto.compta.chaparral.groupfusion.net; www.pcadvisor.co.ukbrucethink.wordpress.com; www.123rf.com

; gamespot.com

commons.wikimedia.org

; neonumbrella.net

;

discoscicatriz.blogspot.com

; diez.hn

; https://encrypted-tbn3.gstatic.com/images;

hubpages.com; javiercordero.com; pidopaso.blogspot.com;

My special thanx

to Ma.Cristina

Llorente

& Patricia Onganía…!

mariana.ferrarelli@gmail.com