multimodal supported writing

RUNNING HEAD: Multimodal Supported Text 1

Multimodal Supported Writing

Jennifer Deyenberg

University of Calgary

Multimodal Supported Text 2

Multimodal Supported Writing

As writing evolves beyond ink and pencils, the nature of text is exploding to

include multimodal elements of audio, video, images, games and nonlinear structures.

Multimodal writing is networked, collaborative, driven by feedback and not done in

isolation. Multimodal supported writing brings forward the idea of supporting this non-

traditional, broader definition of text. The creation of digital text is more than making

traditional text richer or more visually appealing. It is a new genre of writing which

engages more modes, senses and levels than flat, traditional text. This launches writing

into a new realm of creativity and complexity. These complexities are especially

problematic for learners who have additional needs.

If we are truly supporting learners, we need to examine how to support learners

in the changing pedagogy digital tools have afforded. Appropriate and thoughtful

supports, tools, and careful project design is needed to engage and assist all learners.

Literacy is the flexible and sustainable mastery of a repertoire of practices with the texts

of traditional and new communications technologies via spoken, print, and multimedia

(Luke & Freebody, 2000, p. 9)

These new communications and texts fall into a category labeled as multimodal

texts. Multimodal texts are those that rely on the processing and interpretation of print

information, which blends with visual, audio, spoken, nonverbal, and other forms of


expression produced through a range of different technologies and methods (Anstey &

Bull, 2006).

In the Alberta context, the Ministerial Order on Student Learning is the

fundamental document from Alberta Education (2013) implementing the vision of

Inspiring Education. If we are looking at supporting learners in the use of

transformational tools and processes, this document and the competencies listed within,

provide a vision and guidance for learning and research. One of the founding principles

of the document is education in Alberta is, based on the values of opportunity, fairness,

citizenship, choice, diversity and excellence. (Alberta Education, p. 1) This speaks to the

need to provide tools and opportunities for all learners. Further to this basic principle,

the foundations of employ(ing) literacy and numeracy to create meaning (p.2) and

apply(ing) multiple literacies: reading, writing, mathematics, technology, languages,

media and personal finance (p. 3) speak to the need to focus on literacy and writing, but

also employ an expanded view of literacy. The inclusion of media and technology as

literacies expand our view of writing and therefore how we can support changing digital,

media infused writing through assistive technology.

The model of Multimodal text below from the New London Group (1996, p. 24)

recognizes the bringing together of different modes of creating meaning to produce a

more meaningful text.


Figure 1: Design Elements of Multimodal Text (New London Group, 1996, p. 24)

The five modes of meaning as outlined by the New London Group (1996) provide

a framework to examine how to support learners in this evolving writing medium.

The first mode, linguistic design, brings in the elements of word choice,

vocabulary, structure, and meaning. Creating meaning and communication through

traditional text can be especially difficult for students with writing difficulties. Shepard


(2014) found that students were not only independently capable of creating multimodal

projects, but they communicated more about their learning than during class discussion

or using traditional writing.

As we support students in this new medium In order to be effective users of

digital media, students must know how to write for a multimodal environment; they are

adrift in a world of instant publishing without the skills of proficient writers and thinkers

(Klages & Clark, p. 33). Even though writing tools and mediums are changing there is still

a need for process and supporting all learners through that changing process. Instruction

of writing needs to adapt to changing publication speed and methods. We need to

examine how learners who require the most support fit into this new world of writing

and instant publication.

The Hemingway writing tool (http://www.hemingwayapp.com/) is a very nice

web based, instant publication tool to help support word and sentence analysis. It can

help with more text creation by adding colour based analysis of passive voice, readability

level, sentence complexity and other linguistic elements. When you see what elements

are a part of your writing, both positively and negatively you can edit and improve it.

Audio design of a multimodal text gives consideration to sound elements. For

students with a visual impairment, audio based stories can open the door to an

otherwise inaccessible medium. Audio elements in multimodal stories allow for this

accessibility. Adding synthesized audio elements can give students a voice. Broun (2009)

studied students that are nonverbal and how keyboards linked with speech capabilities


can further enhance both writing and communication skills by allowing students to write

and then speak thoughts and contributions. With text to speech tools students can add

their voice and bring their writing alive with multimodal elements.

SoundCloud and Vocaroo are both excellent simple recording and sharing tools.

Students can record audio and save them it in a variety of file types to embed into other

projects or as standalone audio recordings. SoundCloud (https://soundcloud.com/)

works well on mobile devices for recording. Vocaroo (http://vocaroo.com/) works well in

a web browser with one click simple recording.

Kaizena is a feedback tool that can collaboratively add sound, links, images and

feedback to writing. The addition of the Kaizena tool allows students with writing,

processing or visual problems to be able to add and receive comments or add audio

elements to writing.

Writing can have expanded depth, movement, and visual appeal when spatial

elements are considered. Learning to write in part means learning how to produce well-

designed print and digital texts, and learning how the design of the page contributes to

rhetorical import (Archer, 2012, p. 412). The visual spatial elements of writing,

emphasis, materiality, spelling and typography, and how to teach students to consider

these elements needs to be considered as we work with learners with support needs.

Students with print processing difficulties may excel in an environment with more visual

elements. If we work on creating text with more visual elements it can be more

accessible.


A tool such as Prezi (https://prezi.com/) allows for text, image, video, and sound

to be grouped and scaled. Prezi adds movement and zoomable options. These are

unique features to give writing more dynamic elements. Learners can create and express

themselves in a very visual and creative way. For learners with trouble writing words

visuals can be put together in a path to tell a story. The path can be linear or can vary in

size and movement can be added. Because Prezi is visually intensive, it may be a good

choice for students with a hearing impairment. These visual element options can help

create mood or setting that might be restricted when trying to create a video or other

audio rich multimedia writing project.

The story Bear 71 (http://bear71.nfb.ca) from the National Film Board website is

an excellent example of a multimodal story with strong spatial elements. Students can

move through the story with a choice of webcam, keyboard, or mouse making it very

accessible. The story moves following the geolocated movement of the bear, but users

can explore the area around the bears location as well. It combines video, photo, map

and text elements.

Visual design of multimodal elements brings consideration of colour, background,

and perspective. These are engaging elements, but can be difficult to create. Faux (2005)

found using new tools provided an increased cognitive load, but the interest in the tool

and support capabilities of spell check, keyboarding, and visual elements brought more

meaning and ease to the composition process. Pre and post samples of writing indicated

that students were choosing to employ the new literacies developed through introducing


alternative writing methods. The ability to create interactive environments necessitates

a consideration of abstract concepts such as time, space and reader intervention (Faux,

2005, p. 180). This enhancement of skills for learners with additional needs supports the

use of multimodal text in the classroom.

Mindmaps are an excellent visual design tool. They allow for the addition of

pictures, colour and alternate organization structures. Mindmup

(http://www.mindmup.com/) is an online mindmapping tool. It allows for creative,

nonlinear, collaborative multimodal writing across the cloud on a variety of tools.

Gestural design has become more important to the multimodal tools with the

introduction of more touch and gesture based devices such as tablets, smartphones and

video games. Movement can add accessibility and interest to writing. Tapestry

(https://readtapestry.com/) is an app for creating stories which require tapping and

gesture to progress through the story. Adding movement and gesture brings an

interactive and playful element to the writing. Vasudevan (2010) examines the addition

of digital design elements to writing. These elements bring more creativity and play into

the writing process, making it more inviting and more accessible.

Multimodal writing, combining and supporting linguistic, gestural, audio, visual,

and spatial elements, can support students as they create and communicate in the

classroom. In my role I work with teachers and administrators to help support the needs

of students through technology. Multimodal writing means providing more options for

students, but also more considerations as I introduce and recommend tools to support


learners. Jocuis (2013) found that when examining adolescents multimodal retellings

with literary devices, the choice of compositional tool has an important influence on the

types of modes students employ. A large part of my role is helping schools and teachers

select devices and tools to assist with creation. Tools which give access to all modes are

important to support and give access to. As the multimodal medium evolves and new

tools are introduced more creativity options will be available.

Jocuis (2013) goes on to share the emphasis on different modes within

compositions ultimately results in the creation of varying tones and moods, and music

plays an important role in constructing meaning (p. 322). Multimodal writing is an

emerging and exciting medium to support our learners. Students need support to

compose often, compose well, and through these composings become the citizen

writers of our country, the citizen writers of our world, and the writers of our future

(Yancey, 2009, p. 1).


References

Alberta Education. (2013). Ministerial order on student learning. Retrieved from

http://education.alberta.ca/media/6950988/mostudentlearning.pdf

Anstey, M., & Bull, G. (2006).Teaching and learning multiliteracies: changing times,

changing literacies. Newark, DE: International Reading Association.

Archer, A. A. (2012). Writing as design: Enabling access to academic discourse in a

multimodal environment. South African Journal Of Higher Education, 26(3), 411-

421.

Broun, L. (2009). Take the pencil out of the process. TEACHING Exceptional

Children, 42(1), 14-21.

Cho, K., & MacArthur, C. (2010). Student revision with peer and expert reviewing.

Learning and Instruction, 20(4), 328-338.

Faux, F. (2005). Multimodality: How students with special educational needs create

multimedia stories. Education, Communication & Information, 5(2), 167-181.

Jocius, R. (2013). Exploring adolescents' multimodal responses to The Kite Runner:

Understanding how students use digital media for academic purposes. Journal Of

Media Literacy Education, 5(1), 310-325.

Klages, M., & Clark, J. (2009). New worlds of errors and expectations: Basic writers

and digital assumptions. Journal of Basic Writing (CUNY), 28(1), 32-49.

Luke, A. & Freebody, P. (2000). Literate Futures: Report of the Review for

Queensland State Schools, Education Queensland, Brisbane.


New London Group. (1996). A pedagogy of multiliteracies: Designing social futures.

Harvard Educational Review, 66(1), 60-92. Retrieved from

http://vassarliteracy.pbworks.com/f/Pedagogy+of+Multiliteracies_New+London+

Group.pdf

Shepard, A. T. (2014, January 28). Student-generated digital media to support the

literacy skills of high school students with autism spectrum disorder. [Web log

post]. Retrieved from http://leadspartnership.ca/theconversation/student-

generated-digital-media-supports-literacy-autism-spectrum-disorder/

Vasudevan, L.. (2010). Literacies in a participatory, multimodal world: The arts and

aesthetics of web 2.0. Language Arts, 88(1), 43-50.

Yancey, K. B. (2009). Writing in the 21st century. Retrieved from

http://www.ncte.org/library/NCTEFiles/Press/Yancey_final.pdf

multimodal supported writing

Documents