Harnessing the Potential of Engaging With and Comparing Transmedial Narratives for Enhancing Young Learners’ Multimodal Awareness

Speakers  Emilia Djonov (Macquarie University) and Chiao-I Tseng (University of Bremen)

Abstract:

A large body of research has established the need for systematic investigations into learners’ ever more pervasive engagement with new technologies and their competence in accessing, interpreting and authoring multimedia texts (cf. Buckingham 2003, Finch & Arrow 2017; Marsh et al. 2017). As a step towards addressing this need, we consider the question of how engagement with different media can be effectively harnessed for teaching multimodal literacy. 

Inspired by studies that recognise the value of studying ‘transmedial narratives’ for both multimodality (e.g. Tseng & Bateman 2018) and critical multiliteracies education (e.g. Unsworth  2014), in this presentation we examine the potential of engaging with adaptations of the same narrative in different media to build on young children’s awareness of the affordances of different media and the modes they support. Specifically, we integrate:

1. selected results of a study in which 16 dyads of university-educated mothers and their 4-5-year-old children, each read the narrative The Fantastic Flying Books of Mr. Morris Lessmore as an interactive picture book app (Moonbot Studios 2011) and as a traditional-format picture book (Joyce 2012), and
2. the systematic, comparative multimodal analysis of action and event development (based on methods presented in Tseng 2013, 2018) in three versions of that narrative: the award-winning animated short film (Joyce 2011), the book app and traditional-format book.

We argue that comparing basic aspects of the materiality (e.g. presence or absence of interactive hotspots) and semantics (e.g. constructions of events; depictions of and interactions between characters) of adaptations of the same narrative in different media is a method that can help young children enhance their existing multimodal awareness, and support teachers in both evaluating transmedia narratives and later adopting and introducing, in age-appropriate ways, a metalanguage for discussing more complex semantic systems (e.g. ‘focalisation’ – see Unsworth, 2014).

Biosketches:

Emilia Djonov is a Senior Lecturer at Macquarie University, Sydney, Australia, and specialises in language and literacy in early childhood, multiliteracies, multimodal and critical discourse analysis, social semiotics and educational linguistics. In independent and collaborative studies, Emilia has examined the interaction between children’s website design and navigation, and between semiotic technologies (especially ubiquitous communication software) and their employment in various social practices (e.g. children’s interaction with e-games; PowerPoint-supported presentations; integration of websites and learning apps in classrooms). Her research has contributed to critical multimodal discourse studies and research on young children’s multimodal literacy development, language learning and engagement with new technologies.

Chiao-I Tseng is a research associate in the Faculty of Linguistics and Literary Sciences at the University of Bremen. Her area of research focuses on linguistics-informed methods for analysing formal and narrative structures in audiovisual and graphic media, such as methods of cohesion, events structures, narrative space, characters' development and motivations in film, comics, graphic novels and interactive video games. Chiao-I has published journal articles and book chapters on applying these methods to addressing a wide variety of social, cultural and aesthetic issues such as complex film narratives, film genre comparison, authorship, persuasive function in film and graphic novels, screen violence, transmedial comparison, adaptation, narrative space, multimedia and technology in film, etc.

References

Buckingham, D. (2003). Media Education: Literacy, Learning and Contemporary Culture. Cambridge, UK/ Malden, MA, USA: Polity Press.

Finch, B., & Arrow, A. W. (2017). Digital technologies in the literate lives of young children. In C. J. McLachlan & A. W. Arrow (Eds.), Literacy in the Early Years: Reflections on International Research and Practice (pp. 221-238). Singapore: Springer Singapore.

Joyce, W. (Producer), & Joyce, W., & Oldenburg, B. (Directors). (2011). The Fantastic Flying Books of Mr Morris Lessmore [Motion Picture]. USA: Moonbot Studios.

Joyce, W. (2012). The Fantastic Flying Books of Mr. Morris Lessmore (Illustrations by William Joyce and Joe Bluhm). New York: Moonbot Books.  

Marsh, J., Hannon, P., Lewis, M., & Ritchie, L. (2017). Young children’s initiation into family literacy practices in the digital age. Journal of Early Childhood Research, 15(1), 47-60. doi:doi:10.1177/1476718X15582095

Moonbot Studios (2011). The Fantastic Flying Books of Mr Morris Lessmore (Version 1.4). [Mobile Application Software].

Tseng, C.-I. (2013). Analysing characters' interactions in filmic text: a functional semiotic approach. Social Semiotics, 23(5), 587-605. doi:10.1080/10350330.2012.752158

Tseng, C.-I. (2018). Unravelling the myth of multiple endings and the narrative labyrinth in Mr. Nobody (2010). In S. Zhao, E. Djonov, A. Björkvall & M. Boeriis (Eds.), Advancing multimodal and critical discourse studies: Interdisciplinary research inspired by Theo Van Leeuwen’s social semiotics (pp. 131-145). London/New York: Routledge.

Tseng, C., & Bateman, J. A. (2018). Cohesion in comics and graphic novels: an empirical comparative approach to transmedia adaptation in City of Glass. Adaptation.

Unsworth, L. (2014). Point of view in picture books and animated film adaptations: Informing critical multimodal comprehension and composition pedagogy. In E. Djonov & S. Zhao (Eds.), Critical Multimodal Studies of Popular Discourse (pp. 202-216). London/New York: Routledge.