Video games in a multimodal perspective (Reflections on Multimodal semiotics and rhetoric in videogames by Jason Hawreliak)
DOI: 10.23951/2312-7899-2025-1-181-199
Video games, which create and convey meanings through complex semiotic ensembles, are one of the most multimodal media. However, the multimodal methodology per se is hardly ever used in the analysis of video games in the Russian game studies discourse. The article aims to introduce a multimodal approach to the Russian game studies discourse by presenting the key ideas and conclusions of Jason Hawreliak’s book Multimodal Semiotics and Rhetoric in Videogames. The work of the Canadian media semiotician seeks to solve two major problems: firstly, the development and explication of a theoretical approach to the semiotic definition of video games through the prism of multimodality and rhetorical analysis; secondly, the creation of a theoretical basis for a deep, sustainable dialogue between game studies and cognitive linguistics based on multimodality as a common denominator. Methodologically, Hawreliak uses an approach to multimodality based on discourse analysis and social semiotics. Structurally, Hawreliak’s monograph comprises three parts. The first part considers specific types of semiotic modalities realised in videogame media texts. Hawreliak pays special attention to the explication of procedurality as a particular semiotic mode of video games. Here the author refers to the book Hamlet on the Holodeck: The Future of Narrative in Cyberspace by Janet Murray, which adds novelty to the explication of the language of procedurality. Murray’s work still has heuristic value, containing a number of original insights about the dynamic interplay between cybertext and the process of its reading within the consolidating contour of language. Therefore, its rediscovery in the monograph by Hawreliak is important for contemporary media discourse. In addition, the Canadian media semiotician identifies the non-sensory nature of procedurality as the main factor accounting for its being underexplored within the framework of multimodal studies. The second part of the monograph investigates various modal configurations that result from combining different kinds of semiotic modes of video games. Hawreliak studies the phenomenon of multimodal consonance, i.e. the form of a special organisation of semiotic modes within a single ensemble, when they enhance each other’s influence, creating the effect of “rhetorical multiplication”. He introduces into modal consonance the element of user experience, which turns out to be a factor and the most important constituent of the “empirical rhetoric” of video games. Hawreliak expounds the mechanism of phenomenological persuasion in the video game space through the convergence of semiotic modes. The author proposes changing the traditionally negative understanding of game dissonances. He introduces the concept of “modal irony”, demonstrating how it contributes to the formation of new game options for the user through “positive tension”. In the final part of the book, the author empirically proves that video games are unstable semiotic objects. In the final chapter, which is predictive in character, Hawreliak models hypothetical scenarios of possible convergences of multimodality and promising technologies capable of creating new elements of the semiotic design of video games. In terms of style, the work strikes a good balance between comprehensibility and academic rigour. Hopefully, the consideration of multimodality in video games through the analysis of Hawreliak’s book Multimodal Semiotics and Rhetoric in Videogames will contribute to the practical convergence of game studies and multimodal studies.
Keywords: video games, multimodality, Hawreliak, game studies, semiotics of video games, cybertext, semiotic modes of video games
References:
Abousnnouga, G., & Machin, D. (2008). The Visual Institutionalization of Discourses in War Monuments. In A. Mayr (Ed.), Language and Power: An Introduction to Institutional Discourse (pp. 115–137). Continuum.
Belov, S. I. (2022). Historical symbolism of video games as a factor of correction of social memory of the population (on the materials of Assassin’s Creed franchise). Dialog so vremenem – Dialog with Time, 79, 228–235. (In Russian). https://doi.org/10.21267/AQUILO.2022.81.81.017
Belov, S. I. (2024). Reflecting the Cold War in “Atomic Heart”. Science Journal of Volgograd State University. History. Area Studies. International Relations, 29(1), 26–33. https://doi.org/10.15688/jvolsu4.2024.1.3
Belyaev, D. A. (2023). Multimodal and medialinguistic discourse-analytics of video games (Reflections on Astrid Ensslin’s “The Language of Gaming”. Basingstokeand New York: Palgrave Macmillan, 2012. 208 p.). Nauchnyy rezul’tat. Sotsial’nye i gumanitarnye issledovaniya – Scientific Outcome. Social Sciences and Humanities Research, 9(3), 187–193. (In Russian). https://doi.org/10.18413/2408-932X-2023-9-3-0-14
Belyaeva, U. P., & Kopanitsa, V. Yu. (2023). Concept-image of the Soviet Union in the video game “Atomic Heart”: an alternative-historical reconstruction. Sovremennye issledovaniya sotsial’nykh problem – Modern Research on Social Problems, 15(1-3), 29–34. (In Russian).
Bogost, I. (2007). Persuasive Games: The Expressive Power of Videogame. MIT Press. https://doi.org/10.7551/mitpress/5334.001.0001
Boluk, S., & LeMieux, P. (2017). Metagaming: Playing, Competing, Spectating, Cheating, Trading, Making and Breaking Videogames. University of Minnesota Press. https://doi.org/10.5749/j.ctt1n2ttjx
Chernyy, S. Yu. (2017). Constructing the past in video games: the player as consumer and co-author of historical narrative (Europa Universalis 4 project). Shagi – Steps, 3, 77–97. (In Russian).
Ensslin, A. (2012). The Language of Gaming. Palgrave Macmillan.
Forceville, C. (2009). Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Multimodal Metaphor (pp. 19–42). Mouton de Gruyter. https://doi.org/10.1515/9783110197761.5.379
Gee, J. P. (2013). Proactive design theories of sign use: Reflections on Gunther Kress. In Multimodality and Social Semiosis: Communication, Meaning, and Learning in the Work of Gunther Kress (pp. 43–53). Routledge. https://doi.org/10.4324/9780203761090
Hawreliak, J. (2013). Heroism, Gaming, and the Rhetoric of Immortality. Unpublished doctoral dissertation. University of Waterloo, Waterloo, Ontario.
Hawreliak, J. (2018). Multimodal Semiotics and Rhetoric in Videogames. Routledge.
Kromhout, R., & Forceville, C. (2013). Life Is a Journey: Source-Path-Goal Structure in the Videogames “Half-Life 2”, “Heavy Rain”, and “Grim Fandango”. Metaphor and the Social World, 3(1), 100–116. https://doi.org/10.1075/msw.3.1.05for
Moyzhes, L. V. (2020). An Analysis of the Ideological Potential of Video Games from the Point of View of James Gibson’s Theory of Affordances. Sotsiologiya vlasti – Sociology of Power, 32(3), 32–52. (In Russian). https://doi.org/10.22394/2074-0492-2020-3-32-52
Murray, J. (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace. Simon and Schuster.
Norris, S. (2004). Analyzing Multimodal Interaction: A Methodological Framework. Routledge.
Pstyga, A. (2021). Multimodality of media text in the new communicative space (observations and remarks). Voprosy zhurnalistiki, pedagogiki, yazykoznaniya – Issues of Journalism, Pedagogy, Linguistics, 40(4), 442–450. (In Russian).https://doi.org/10.52575/2712-7451-2021-40-4-442-450
Sutton-Smith, B. (2001). The Ambiguity of Play. Harvard University Press.
Toh, W. (2015). A Multimodal Discourse Analysis of Video Games: A Ludonarrative Model. Unpublished doctoral dissertation. National University of Singapore.
Toh, W. (2019). A Multimodal Approach to Video Games and the Player Experience. Routledge.
Tyazhlova, Y., Manokhin, D., Kozhemyakin, E., & Srybnyy, D. (2021). Multimodal Rhetoric of Acute Media Text. Media Education (Mediaobrazovanie), 17(1), 144–152. https://doi.org/10.13187/me.2021.1.144
Wilcox, S. (2014). Procedural Realism: The Political Representation of Reality in Videogames. First Person Scholar. http://www.firstpersonscholar.com/procedural-realism/
Zagidullina, M. V. (2023). Modern state of multimodal analysis: to the question about the prospects of the method. Nauchnyy rezul’tat. Sotsial’nye i gumanitarnye issledovaniya – Scientific Outcome. Social Sciences and Humanities Research, 9(1), 84–99. (In Russian). https://doi.org/10.18413/2408-932X-2023-9-1-0-7
Zagidullina, M. V. (2023) Modern state of multimodal analysis: to the question about the prospects of the method. Scientific Outcome. Social Sciences and Humanities Research, 9 (1), 84–99. doi: 10.18413/2408-932X-2023-9-1-0-7 (In Russian).
( 364.37 kB ) 
( 351.33 kB )
Issue: 1, 2025
Series of issue: Issue 1
Rubric: BOOK REVIEWS
Pages: 181 — 199
Downloads: 1104