Multimodal literacy learning with interactive technologies Sue Nichols Centre for Research in Education

Multimodality The term ‘multimodal’ is used to refer to a text constructed from different kinds of meaning- making resources eg print language, sound & visual images. Some resources used by text producers:

Multimodality The term ‘multimodal’ is used to refer to a text constructed from different kinds of meaning- making resources eg print language, sound & visual images. Some resources used by text producers: Graphic symbols Spoken language Sound effects Moving images Hyper-linking Concrete materials Written language

Sally (aged 4) uses visual resources CONTEXT: Sally had noticed that a researcher visiting her kindergarten was writing in a notebook. She asked to draw something in the notebook. A yoghurt container full of coloured feathers was on the table. Sally announced she would draw that and asked if the researcher would draw the container first. Then the following conversation happened …

Sally uses spoken language and social resources Sally: Sally has green feathers and red feathers and purple feathers and yellow feathers and purple flowers. You can write it in your story now. Sue: OK. Sally: I’m going to draw a feather. I’ll do the purple one. You do the next one. Sue: OK. I’ll do the green one. Sally: You do it like that. These are the fluffy bits. You have to go down to here. You have to be careful with textas. You’re doing it nicely, aren’t you? Sue: How’s that? Sally: Good. I’ll do the next one.

Sally employs written language resources via an expert helper. Here, the adult helper is the resource, or indeed the technology, which is utilised by Sally to produce the written part of her text.

Multimodality is not new However, digital technologies offer a different set of resources for producing and consuming multimodal texts. They can make it easier to work across modes and to include modes that have become neglected.

Digital word processing has been changing the nature and pace of children’s writing development One of the children had actually sent messages in September with spaces between words, but did not begin to consistently add spaces in her handwriting for 18 months. One child typed his name, “Jeffery”, highlighted it, selected the WingDing font type, and gleefully exclaimed, “I made my name in code.” The typed name “Jeffery” had become  Yost, N. (2003) Computers, Kids, and Crayons: A comparative study of emergent writing behaviours. IFIP Working Group 3.5 Conference: Young Children and Learning Technologies, held at UWS Parramatta

Digital technologies can create different opportunities for children’s communication. New audiences New forms of written communication New kinds of collaboration Students becoming ‘prosumers’

Collaborative online writing In the United States, Common Core State Standards (CCSS) requires teachers to “engage students in collaboration and online publication of writing”. Teachers Mary Blow and Ronica Lawrence, working in schools over 100k apart, decided to partner their classes in an online science research project. accessed 22/8/12

Online collaboration using Knowledge Forum Many children were motivated to make entries themselves using whatever strategies they had available, including use of invented spelling. Pelletier et al 2012 Children could see the photographs and the ideas posted by their peers. By clicking on a peer’s entry, text scaffolds in the database would appear that gave children choices; for example: “my theory is…” or “I want to know.”

Integrating reading & writing with digital devices don’t worry Humphrey i have a terrible life with my sister i wonder what garths house looked like what does that line mene that is werd Sourced from Larson, L. (2010) Digital Readers: The next chapter in e-book reading and response. The Reading Teacher 64(1) pp 15 – 22. Amy ( aged 7) & Winnie (aged 8) made notes on their Kindles while reading Friendship According to Humphrey. Amy made 43 notes and Winnie made 33.

Literacy Apps – Blending old and new First Published 1979 This app is a version of a class children’s book. It is not unusual to see older children’s literature appearing in app form, as it is cheaper for producers than employing authors to create original content. Given what Susan Hill’s research is revealing about the popularity of mobile phones with very young children, this offers a way to engage them with quality children’s literature at little cost to parents.

What has changed? How many of these functions could be carried out without digital technology? There are significant continuities in so-called ‘New Literacy’ texts and practices.

Personal Examples E-book making in libraries Photo studio in classrooms In the following slides I share some examples of multimodal texts produced by children in two projects I have been involved in.

E-book Making in Libraries Sessions for children and carers were run in school holiday programs at two metropolitan libraries. Separate sessions were run for younger (aged 3 – 6) and older (aged 7 – 10) children. Sessions were 90 minutes long and utilised the iPad app ‘My Story’. Sessions began with a ‘hands off’ pre-production phase. PRE-PRODUCTION: 1.Icon recognition (It is vital for users to understand how to access functions like ‘undo.’) 2.Explanation of multiple modes & non-linear production (Users can commence creating their story with drawing, photography, writing or oral narration.) 3.Narrative planning ( A ‘hands-off’ time for children and their carers to decide on characters and story line. Deciding on a title is a good focusing move.)

PRODUCING THE E-BOOK RESOURCES: iPads paper & textas puppets & figurines large pieces of coloured cardboard as backdrops cubes for setting up scenes PROCESS: Most children began with the visual element. Children chose to either draw or to stage and photograph scenes using props. Some combined both. The language part of the narrative was either dictated to an adult to type or was recorded as an oral narration by the child. Some children chose to type their own text. Narratives that had been well planned could be produced more efficiently since children generally moved quickly to the complication part of the story.

NARRATED BY CHILD: Once there was a princess

NARRATED BY CHILD: Once the princess met a dragon.

NARRATED BY CHILD: The dragon ate the princess.

NARRATED BY CHILD: He didn’t like her and he spat her out.

Created on an iPad by ‘Jack’, a Junior Primary child, using photography, drawing, handwritten text & text typed by an adult but no oral narration. Once there was a fire breathing dragon

One day the dragon found that he couldn’t breath fire any more.

So the dragon went to see the giraffe.

The giraffe waved his magic wand.

Suddenly the dragon could breathe fire again. The end.

Classroom Photo Studio: Year 1 Children making design choices For this project, children worked with digital cameras and PowerPoint to produce representations of significant personal objects. A photo booth was set up in the classroom with a digital camera, tripod, table and display board. Cardboard provided the backdrop. PROCESS: Children posed and photographed their objects. The photographs were saved to the class computer. Children then took turns to dictate written text in one or more of three formats: – Speech bubble – Thought bubble – Caption All the slides were combined in a class presentation that was shown during the school’s Art Night.

Eagles Aaaah!! Eagles are scary. DESIGN DECISIONS: What object to show; How to pose it; Whether to add text; What text to add; Speech or thought bubble or caption; Colour for text and fill.

My car knows everything about the world. My car has been all over the world Wheeeeee!!!

Images of his toy shark taken by a 5 year old boy Adult: Is this how you want it to look? Child: Shakes head Adult: How do you want it to look? Child: Fierce. Image selected by child. Propping shark on plastic cup to show mouth.

Questions about any technology Is it flexible? Can it be used in combination with existing tools? Does it foster the integration of skills? Does it facilitate both scaffolded and independent use by children? Can it be used for text production as well as for the consumption of texts produced by others? Does it support different ways of learning and of displaying learning?

REFERENCES Larson, L. (2010) Digital Readers: The next chapter in e-book reading and response. The Reading Teacher 64(1) pp 15 – 22. Pelletier, J., Reeve, R., & Halewood, C. (2006). Young children's knowledge building and literacy development through Knowledge Forum. Early Education and Development, 17(3), Yost, N. (2003) Computers, Kids, and Crayons: A comparative study of emergent writing behaviours. IFIP Working Group 3.5 Conference: Young Children and Learning Technologies, held at UWS Parramatta and-online-publishing accessed 22/8/12 and-online-publishing TO REFERENCE THIS PRESENTATION: Nichols, S. (2013) Multimodal literacy learning with interactive technologies. Presentation for the Centre for Research in Education, University of South Australia, Adelaide, July 3 rd 2013.