The Role of Semiotic Metaphor in the Verbal-Visual Interplay of Three Children’s Picture Books. A Multisemiotic Systemic-Functional Approach
Abstract
This paper aims to explore how the use of semiotic metaphors in picture books contributes to children’s understanding of the stories. The three picture books selected for analysis were written during the twentieth century and respond to a standard of literary quality: Guess How Much I Love You (1994), Where the Wild Things Are (1963) and Gorilla (1983). The concept of semiotic metaphor as a tool to create ideational meaning is analysed within the framework of Systemic Functional Linguistics and Systemic-Functional Multimodal Discourse Analysis. Kay O’Halloran extends the Hallidayan concept of grammatical metaphor to the semiotic metaphor in order to determine how verbal and visual modes interact with each other in multimodal texts. Like grammatical metaphor, semiotic metaphor also involves a shift in the grammatical class or function of an element. As this process does not take place intra-semiotically, but rather inter-semiotically, the reconstrual produces a semantic change in the function of that element, creating a new way of making meaning and representing reality. The results of the analysis show that semiotic metaphors are essentially used in children’s tales to facilitate young children’s understanding of the story by making some abstract phenomena related to states of being more concrete and specific. Keywords: Systemic Functional Linguistics; grammatical metaphor; semiotic metaphor; verbal-visual intersemiosis; picture booksReferences
Agosto, Denise E. 1999. “One and Inseparable: Interdependent Storytelling in Picture Storybooks.” Children’s Literature in Education 30 (4): 267-280.
Barthes, Roland. 1977. Image-Music-Text. Translated by Stephen Heath. London: Fontana.
Bateman, John. 2014. Text and Image. A Critical Introduction to the Visual/Verbal Divide. London and New York: Routledge.
Browne, Anthony. (1983) 2002. Gorilla. London: Walter Books.
Butt, David, Rhondda Fahey, Susan Feez, Sue Spinks and Colin Yallop. 2003.Using Functional Grammar. An Explorer’s Guide. 2nd ed. Sydney: National Centre for English Language Teaching and Research, Macquarie University.
Caballero, Rosario. 2009. “Cutting across the Senses: Imagery in Winespeak and Audiovisual Promotion.” In Multimodal Metaphor, edited by Charles Forceville and Eduardo Urios-Aparisi, 73-94. Berlin and New York: Mouton de Gruyter.
Cerrillo, Pedro and Santiago Yubero. 2007. “Qué leer y en qué momento” [What to Read and When]. In La formación de mediadores para la promoción de la lectura [Training Specialists in the Promotion of Reading], edited by Pedro Cerrillo and Santiago Yubero, 285-293. Cuenca: Servicio de Publicaciones de la Universidad de Castilla-La Mancha.
Coffin, Caroline, Jim Donohue and Sarah North. 2009. Exploring English Grammar. From Formal to Functional. London and New York: Routledge.
Cohn, Neil. 2013. “Beyond Speech Ballons and Thought Bubbles: The Integration of Text and Image.” Semiotica 197: 35-63.
Downing, Angela. 2014. English Grammar. A University Course. 3rd ed. London and New York: Routledge.
Downing, Angela and Philip Locke. 2006. English Grammar. A University Course. 2nd ed. London and New York: Routledge.
Forceville, Charles. 1996. Pictorial Metaphor in Advertising. London: Routledge.
Forceville, Charles and Eduardo Urios-Aparisi, eds. 2009. Multimodal Metaphor. Berlin and New York: Mouton de Gruyter.
Gamble, Nikki and Sally Yates. 2002. Exploring Children’s Literature: Teaching the Language and Reading Fiction. London: Paul Chapman.
Halliday, M.A.K. 1978. Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.
—. (1985) 1994. An Introduction to Functional Grammar. London: Edward Arnold.
—. 1998. “Things and Relations.” In Reading Science: Critical and Functional Perspectives on Discourses of Science, edited by Jim R. Martin and Robert Veel, 185-235. London and New York: Routledge.
—. 2004. An Introduction to Functional Grammar. 3rd ed., revised by Christian M.I.M. Matthiessen. London: Edward Arnold.
—. 2014. Halliday’s Introduction to Functional Grammar. 4th ed., revised by Christian M.I.M. Matthiessen. London and New York: Routledge.
Halliday, M.A.K. and Christian M.I.M. Matthiessen. (1999) 2006. Construing Experience through Meaning. A Language-Based Approach to Cognition. 5th ed. London: Equinox.
Halliday, M.A.K. and Jonathan J. Webster. 2014. Text Linguistics. The How and Why of Meaning. London: Equinox.
Hidalgo-Downing, Laura and Blanca Kraljevic Mujic. 2011. “Multimodal Metonymy and Metaphor as Complex Discourse Resources for Creativity in ICT Advertising Discourse.” Review of Cognitive Linguistics (Special issue on Metaphor and Metonymy) 9 (1): 153-178.
Hunt, Peter, ed. 2004. International Companion Encyclopaedia of Children’s Literature. 2nd ed. London: Routledge.
Jewitt, Carey, ed. 2009. The Routledge Handbook of Multimodal Analysis. London and New York: Routledge
Kong, Kenneth C. 2006. “A Taxonomy of the Discourse Relations between Words and Visuals.” Information Design Journal 14 (3): 207-230.
Kress, Günther, 2010. Multimodality. A Social Semiotic Approach to Contemporary Communication. London and New York: Routledge.
Kress, Günther and Theo van Leeuwen (1996) 2006. Reading Images. The Grammar of Visual Design. London: Routledge
—. 2001. Multimodal Discourse. The Modes and Media of Contemporary Communication. London: Edward Arnold.
Lakoff, George. 1987. Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. Chicago: U of Chicago P.
—. 1993. “The Contemporary Theory of Metaphor.” In Metaphor and Thought, edited by Andrew Ortony, 202-251. Cambridge: Cambridge UP.
Lakoff, George and Mark Johnson. 1980. Metaphor We Live By. Chicago: Chicago UP.
Lemke, Jay. 1998. “Multiplying Meaning: Visual and Verbal Semiotics in Scientific Texts.” In Reading Science: Critical and Functional Perspectives on Discourses of Science, edited by Jim R. Martin and Robert Veel, 87-113. London: Routledge.
Liu, Yin and Kay O’Halloran. 2009. “Intersemiotic Texture: Analysing Cohesive Devices between Language and Images.” Social Semiotics 19 (4): 367-388.
Mann, William C. and Sandra A. Thompson. 1988. “Rhetorical Structure Theory: Towards a Functional Theory of Text Organization.” Text 8 (3): 243-281.
Martin, Jim. 1992. English Text: System and Structure. Amsterdam and Philadelphia: John Benjamins.
—. 2008. “Incongruent and Proud: De-vilifying Nominalisation.” Discourse & Society 19(6): 801-810.
Martin, Jim and D. Rose. 2003. Working with Discourse: Meaning beyond the Clause. London: Continuum.
Martinec, Radan and Andrew Salway. 2005. “A System for Image-Text Relations in New (and Old) Media.” Visual Communication 4 (3): 337-371.
McBratney, Sam and Anita Jeram. 1994. Guess How Much I Love You. London: Walter Books.
McQuarrie, Edward F. and David G. Mick. 1999. “Visual Rhetoric in Advertising Language.” Journal of Consumer Research 22 (4): 424-438.
Moya-Guijarro, A. Jesús. 2014. A Multimodal Analysis of Picture Books for Children. A Systemic Functional Approach. London: Equinox.
Moya-Guijarro, A. Jesús and Jesús A. Ávila. 2009. “Thematic Progression of Children’s Stories as Related to Different Stages of Cognitive Development.” Text and Talk 29 (6): 755-774.
Moya-Guijarro, A. Jesús and María Jesús Pinar. 2009. “On Interaction of Image and Verbal Text in a Picture Book. A Multimodal and Systemic Functional Study.” In The World Told and the World Shown: Multisemiotic Issues, edited by Eija Ventola and A. Jesús Moya-Guijarro, 107-123. London and New York: Palgrave Macmillan.
Nikolajeva, Maria and Carol Scott. 2001. How Pictures Work. London: Routledge.
O’Halloran, Kay. 1999. “Interdependence, Interaction and Metaphor in Multisemiotic Texts.” Social Semiotics 9 (3): 317-354.
—. 2000. “Classroom Discourse in Mathematics: A Multisemiotic Analysis.” Linguistics and Education 10 (3): 359-388.
—. 2003. “Intersemiosis in Mathematics and Science: Grammatical Metaphor and Semiotic Metaphor.” In Grammatical Metaphor: Views from Systemic Functional Linguistics, edited by Anne Marie Simon-Vandenbergen, Miriam Taverniers and Louise J. Ravelli, 337-365. Amsterdam: John Benjamins.
—. 2004. Multimodal Discourse Analysis. London: Continuum.
—, ed. 2005. Mathematical Discourse: Language, Symbolism and Visual Images. London: Continuum.
—. 2007. “Systemic Functional Multimodal Discourse Analysis (SF-MDA) Approach to Mathematics, Grammar and Literacy.” In Advances in Language and Education, edited by Anne McCabe, Mick O’Donnell and Rachel Whittaker, 77-102. London: Continuum.
—. 2008. “Systemic Functional Discourse Analysis (SF-MDA): Constructing Ideational Meaning Using Language and Visual Imagery.” Visual Communication 7 (4): 443-475.
Painter, Claire, Jim Martin and Len Unsworth. 2013. Reading Visual Narratives: Image Analysis of Children’s Picture Books. London: Equinox.
Phillips, Barbara J. and Edward F. McQuarrie. 2004. “Beyond Visual Metaphor: A New Typology of Visual Rhetoric in Advertising.” Marketing Theory 4 (1/2): 113-136.
Pinar, María Jesús, ed. 2013. Review of Cognitive Linguistics (Special issue on Multimodality and Cognitive Linguistics) 11 (2).
—. 2015. Multimodality and Cognitive Linguistics. Amsterdam and Philadelphia: John Benjamins.
Ravelli, Louise. 1985. “Metaphor, Mode and Complexity: An Exploration of Co-Varying Patterns.” BA Dissertation. Unpublished BA (Hons) Thesis. University of Sydney.
—. 1988. “Grammatical Metaphor: An Initial Analysis.” In Pragmatics, Discourse and Text. Inspired Approaches, edited by Erich H. Steiner and Robert Veltman, 133-148. London: Pinter.
Sendak, Maurice. (1963) 2007. Where the Wild Things Are. London: Red Fox.
Stöckl, H. 2004. “In Between Modes: Language and Image in Printed Media.” In Perspectives on Multimodality, edited by Eija Ventola, Cassily Charles and Martin Kaltenbacher, 9-30. Amsterdam: John Benjamins.
Taverniers, Miriam. 2014. “Grammatical Metaphor and Grammaticalisation: Fractal Patterns in Linguistic Change.” Paper presented at the 25th European Systemic Linguistic Conference. Paris: U Paris Diderot. July, 2014.
Toboada, Maite and Christopher Habel. 2013. “Rhetorical Relations in Multimodal Documents.” Discourse Studies 15 (1): 65-89.
Townsend, J. R. 1990. “Standards of Criticism for Children’s Literature.” In Children’s Literature: The Development of Criticism, edited by Peter Hunt, 57-70. London: Routledge.
Turner, Mark. 1996. The Literary Mind. Oxford: Oxford UP.
Unsworth, Len. 2007. “Image/Text Relations and Intersemiosis: Towards Multimodal Text Description for Multiliteracies Education.” In Proceedings of the 33rd International Systemic Functional Congress, edited by Leila Barbara and Tony Berber Sardinha, 1165-1205. Sao Paulo: Pontifícia Universidade Católica de Sao Paulo.
Unsworth, Len and Chris Cléirigh. 2009. “The Construction of Meaning through Image-Text Interaction.” In The Routledge Handbook of Multimodal Analysis, edited by Carey Jewitt, 151-165. London: Routledge.
Van Leeuwen, Teo. 1991. “Conjunctive Structure in Documentary Film and Television.” Continuum: Journal of Media and Cultural Studies 5 (1): 76-114.
—. 2005. Introducing Social Semiotics. London: Routledge.
The authors retain copyright of articles. They authorise AEDEAN to publish them in its journal Atlantis and to include them in the indexing and abstracting services, academic databases and repositories the journal participates in.
Under the terms of the Creative Commons Attribution NonCommercial ShareAlike 4.0 International Licence (CC BY-NC-SA 4.0), for non-commercial (i.e., personal or academic) purposes only, users are free to share (i.e., copy and redistribute in any medium or format) and adapt (i.e., remix, transform and build upon) articles published in Atlantis, free of charge and without obtaining prior permission from the publisher or the author(s), as long as they give appropriate credit to the author, the journal (Atlantis) and the publisher (AEDEAN), provide the relevant URL link to the original publication and indicate if changes were made. Such attribution may be done in any reasonable manner, but not in any way that suggests the journal endorses the user or their use of the material published therein. Users who adapt (i.e., remix, transform or build upon the material) must distribute their contributions under the same licence as the original.
Self-archiving is also permitted, so that authors are allowed to deposit the published PDF version of their articles in academic and/or institutional repositories, without fee or embargo. Authors may also post their individual articles on their personal websites, again on condition that the original link to the online edition is provided.
Authors are expected to know and heed basic ground rules that preclude simultaneous submission and/or duplicate publication. Prospective contributors to Atlantis commit themselves to the following when they submit a manuscript:
- That no concurrent consideration of the same, or almost identical, work by any other journal and/or publisher is taking place.
- That the potential contribution has not appeared previously, in any form whatsoever, in another journal, electronic format or as a chapter/section of a book.
Seeking permission for the use of copyright material is the responsibility of the author.