From Comics Strips to Serious Games: Notes about the Multimodal Complexity of First-Aid Instructions

Janina Wildfeuer, University of Groningen

Abstract

CPR, the Heimlich maneuver, or methods such as checking the pulse and securing the head and neck are important life-saving procedures that everyone should know when involved in emergency situations. How to follow the necessary steps and perform urgent measures is therefore communicated in first-aid instructions in the form of leaflets, posters, or info screens. This multimodal text genre nowadays not only consists of verbal documents, but also includes sequential-pictorial information in comics as well as dynamic audio-visuals such as videos or computer games. As highly complex artifacts, they combine different semiotic modes including writing or speech, static or moving images, layout, gestures, animation, music or sound to make meaning. While the complexity of these instructive forms has increased substantially in the last years, the genre has at the same time adopted and adjusted patterns and structures from other genres such as narratives, for example. As a consequence, these hybrid forms of communication represent highly interesting and innovative research objects for multimodal analysis with far-reaching social implications.

In my talk, I will first give an overview of the diversity and multimodal complexity of the genre of first-aid instructions and point to examples that are particularly challenging for multimodal analysis. I will then make suggestions how to examine and compare the various artifacts systematically and with methodologically integrative and large-scale empirical investigations.

Bio note

Janina Wildfeuer is an Assistant Professor in the Department of Communication and Information Studies at the University of Groningen, NL. Her main research interests lie in multimodal linguistics, discourse analysis, media studies, and semiotics. She teaches classes in multimodal, interdisciplinary, and applied linguistics and analyses films, comics, video games, and other multimodal documents in several projects exploring the notion of multimodal discourse. Her publications include several monographs and edited collections as well as contributions and articles on the analysis of multimodal artifacts, mostly focusing on interdisciplinary approaches in the humanities and beyond.

Register at https://www.eventbrite.co.uk/e/janina-wildfeuer-from-comic-strips-to-serious-games-multimodality-talk-tickets-132279825519

The Multimodality Talks Series is organised by the University of Leeds Multimodality@Leeds, The UCL Visual and Multimodal Research Forum, and Insulander/Svärdemo Åberg at the Department of Education, Stockholm University. It is conceived of working as a tandem with the Bremen-Groningen Online Workshops on Multimodality to make the best of the online format to offer multiple chances for sharing research and stimulating discussions on multimodality worldwide. See the full Multimodality Talk Series programme here: https://mmodalityleeds.wordpress.com/2020/12/11/programmes-multimodality-talks-series/