Multimodal Discourse Analysis and Electronic Devices: A Pilot Study on Emotionally Supported Evidence
DOI:
https://doi.org/10.28914/Atlantis-2024-46.2.04Abstract
Communication is multimodal in its very nature. In recent decades, research interest in this topic has grown exponentially, especially from a Multimodal Discourse Analysis (MDA) standpoint. Within academic settings, several studies have looked at lecturers’ combinations of verbal and non-verbal features, but not necessarily in relation to the emotions that are implicitly part of lecturing. We believe that teaching involves transmitting knowledge together with emotions to students, more or less consciously. This may even be more relevant in an English Medium Instruction (EMI) setting, as English is not the instructor’s main language. Thus, our main aim is to analyse an example of EMI teaching practice from an MDA perspective but adding an electronic device—an electroencephalograph—that can help us improve and/or complement the analysis in different ways: adding more objective support and dealing with the emotions lecturers can transmit when delivering their classes. Results show that this combination of observation and technology can potentially enrich results from traditional MDA research.
Downloads
References
Akay, Metin et al, eds. 2011. Proceedings of the 5th International IEEE/EMBS Conference on Neural Engineering. Cancun, Mexico: IEEE.
Alugupalli, Nripen, Ashok Samal, David Marx and Sanjiv Bhatia. 2011. “Analysis of Landmarks in Recognition of Face Expressions.” Pattern Recognition and Image Analysis 21 (4), 681-93.
Amir, Nader et al. 2008. “The Effect of a Single-Session Attention Modification Program on Response to a Public-speaking Challenge in Socially Anxious Individuals.” Journal of Abnormal Psychology 117 (4), 860-68.
Aucejo-Devís, Bárbara and María Pocoví. 2017. “Experiential Marketing: Searching for Emotional Connection with Consumers in POS.” In Dos Santos 2017, 63-81.
Banks, Matthew. 2020. “‘Seeing is Believing’: Developing EMI Lecturers’ Classroom Interactional Competence Using Video Enhanced Observation.” In González-Álvarez and Rama-Martínez 2020, 100-19.
Benito, Águeda and Ana Cruz, coord. 2005. Nuevas claves para la docencia universitaria en el espacio Europeo de educación superior. Madrid: Narcea.
Bernad-Mechó, Edgar. 2018. “A Multimodal Approach to Metadiscourse as an Organizational Tool in Lectures.” PhD diss., Universitat Jaume I.
—. 2021. “Combining Multimodal Techniques to Approach the Study of Spoken Academic Lectures: A Methodological Reflection.” Atlantis 34 (1): 178-98.
Björkman, Beyza. 2011. “English as a Lingua Franca in Higher Education: Implications for EAP.” Ibérica 22: 79-100.
Blythe, Mark A. et al, eds. 2003. Funology: From Usability to Enjoyment. Berlin: Springer.
Bos, Danny Oude. 2006. “EEG-based Emotion Recognition: The Influence of Visual and Auditory Stimuli.” PhD diss., University of Twente.
Bowles, Hugo and Alessia Cogo, eds. 2015. International Perspectives on English as a Lingua Franca: Pedagogical Insights. London: Palgrave Macmillan.
Cacioppo, John T. et al, eds 2001. “The Psychophysiology of Emotion.” In Lewis and Haviland-Jones 2001, 173-91.
Chanel, Guillaume, Karim Ansari-Asl and Thierry Pun. 2007. “Valence-Arousal Evaluation Using Physiological Signals in an Emotion Recall Paradigm.” In N.a. 2007, 2662-67.
Choppin, Antoine. 2000. “EEG-Based Human Interface for Disabled Individuals: Emotion Expression with Neural Networks.” Master’s thesis, Tokyo Institute of Technology.
Crawford-Camiciottoli, Belinda and Inmaculada Fortanet-Gómez, eds. 2015. Multimodal Analysis in Academic Settings. From Research to Teaching. New York and London: Routledge.
Cuddy, Amy. 2015. Presence: Bringing your Boldest Self to your Biggest Challenges. New York: Little Brown.
Davidson, Richard J., Klaus R. Scherer and H. Hill Goldsmith, eds. 2003. Handbook of Affective Sciences. Oxford: Oxford UP.
Desmet, Pieter M. A. 2003. “Measuring Emotions: Development and Application of an Instrument to Measure Emotional Responses to Products.” In Blythe et al. 2003, 111-23.
Diani, Giuliana. 2008. “Emphasizers in Spoken and Written Academic Discourse: The Case of Really.” International Journal of Corpus Linguistics 13 (3): 296-321.
Doerrfeld, Bill. 2015. “20+ Emotion Recognition APIs that Will Leave You Impressed, and Concerned.” Nordic Apis, December 31. [Accessed January 20, 2018].
Doiz, Aintzane, David Lasagabaster and Víctor Pavón. 2019. “The Integration of Language and Content in English Medium Instruction Courses: Lecturers’ Beliefs and Practices.” Ibérica 38, 151-76.
Dos Santos, Manuel Alonso, ed. 2017. Applying Neuroscience to Business Practice. Hershey, PA: IGI Global.
Ekman, Paul and Wallace V. Friesen. 1978. Facial Action Coding System. Investigator’s Guide. Palo Alto: Consulting Psychologists.
Fortanet-Gómez, Inmaculada. 2013. CLIL in Higher Education. Towards a Multilingual Language Policy. Bristol: Multilingual Matters.
—. 2020. “The Dimensions of EMI in the International Classroom: Training Teachers for the Future University.” In Sánchez-Pérez 2020, 1-20.
González-Álvarez, Dolores and Esperanza Rama-Martínez, eds. 2020. Languages and the Internationalisation of Higher Education. Newcastle Upon Tyne: Cambridge Scholars.
Hamdi, Hamza et al. 2012. “Emotion Assessment for Affective Computing Based on Physiological Responses.” Paper given at the 2012 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Brisbane, June 2012.
Harrison, Tegan. 2013. The Emotiv Mind: Investigating the Accuracy of the Emotiv EPOC in Identifying Emotions and its Use in an Intelligent Tutoring System. Canterbury: University of Canterbury.
Hernández, Roberto and César Cárdenas. 2014. “How Emotions Affect the Learning Process in Interactive Scenarios: Towards Evaluation and Leverage of User’s Affective Conditions in a BCI-based Interactive Module.” In MexIHC ‘14 2014, 42-45.
Hyland, Ken and Brian Paltridge, eds. 2011. Companion to Discourse. London: Continuum.
Kaltenbacher, Martin. 2007. “Perspectivas en el análisis de la multimodalidad. Desde los inicios al estado del arte.” Revista Latinoamericana de Estudios del Discurso 7 (1), 31-57.
Khan, Sarah. 2018. “Lecturing Strategies of Non-Native EMI Lecturers on an International Business Programme.” Bulletin VALS-ASLA 107, 65-82.
Kirkpatrick, Andy, ed. 2001. The Routledge Handbook of World Englishes. London and New York: Routledge.
Kress, Gunther. 1996. Reading Images: The Grammar of Visual Design. London and New York: Routledge.
—. 2001. Multimodal Discourse. The Modes and Media of Contemporary Communication. London: Arnold.
—. 2012. Multimodality: A Social Semiotic Approach to Contemporary Communication. London and New York: Routledge.
— and Theo van Leeuwen. 1990. Reading Images. Geelong: Deakin UP.
Lackovic, Natasa and Biliana Popova. 2021. “Multimodality and Socio-Materiality of Lectures in Global Universities Media: Accounting for Bodies and Things.” Learning, Media and Technology 46 (4), 531-49.
Levenson, Robert W. 2003. “Autonomic Specificity and Emotion.” In Davidson, Scherer and Goldsmith 2003, 212-24.
LeVine, Philip and Ron Scollon, eds. 2004. Discourse and Technology: Multimodal Discourse Analysis. Georgetown: Georgetown UP.
Lewis, Michael and Jeannette M. Haviland-Jones eds. 2001. Handbook of Emotions. 2nd ed. New York: The Guilford Press.
Mauranen, Anna, Carmen Pérez-Llantada and John Swales. 2010. “Academic Englishes: A Standardized Knowledge?” In Kirkpatric 2001, 634-53.
Menéndez, Salvio. 2012. “Multimodalidad y estrategias discursivas. Un abordaje metodológico.” Revista Latinoamericana de Estudios del Discurso 12 (1), 57-73. MexIHC ’14. 2014. Proceedings of the 5th Mexican Conference on Human-Computer Interaction. New York: Association of Computing Machinery.
Morell, Teresa 2018. “Multimodal Competence and Effective Interactive Lecturing.” System 77 (1), 70-79.
Moyers, Bill. 1989. A World of Ideas: Conversations with Thoughtful Men and Women about American Life Today and the Ideas Shaping our Future. New York: Doubleday.
N.a. 2007. Proceedings of the 2007 IEEE International Conference on Systems, Man and Cybernetics. Quebec: Institute of Electrical and Electronics Engineers (IEEE).
—. 2014. Proceedings from the 18th International Congress on Project Management and Engineering. Zaragoza: AEIPRO.
Nie, Dan et al. 2011. “EEG-Based Emotion Recognition during Watching Movies.” In Akay et al. 2011, 667-70.
Norris, Sigrid. 2004. Analyzing Multimodal Interaction. A Methodological Framework. London and New York: Routledge.
O’Connor, Catherine and Catherine Snow. 2018. “Classroom Discourse: What Do We Need to Know for Research and for Practice?” In Schober et al. 2018, 315-42.
O’Halloran, Kay. 2011. “Multimodal Discourse Analysis.” In Hyland and Paltridge 2011, 120-37.
Pantic, Maja and Léon J. M. Rothkrantz 2000. “Automatic Analysis of Facial Expressions: The State of the Art.” IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (12), 1424-45.
Preston, Paul. 2005. “Nonverbal Communication: Do You really Say What You Mean?” Journal of Healthcare Management 50 (2), 83-86.
Puig-Martí, M. Dolors and Adelina Bolta-Escolano. 2014. “La metodología ‘Emotion Research Lab’ en diseño de objeto para resguardar de la lluvia a usuarios en silla de ruedas.” In N.a. 2014, 1976-88.
Quartz, Steven R. 2009. “Reason, Emotion and Decision-Making: Risk and Reward Computation with Feeling.” Trends in Cognitive Science 13 (8), 209-15.
Querol-Julián, Mercedes and Inmaculada Fortanet-Gómez. 2012. “Multimodal Evaluation in Academic Discussion Sessions: How Do Presenters Act and React?” English for Specific Purposes 31, 271-83.
Rahman, Mohammed H. 2018. “Paralinguistic Features in TESOL: An Action-Based Approach.” International Journal of Current Research 10 (10), 74319-22.
Royo-González, Marta. 2016. “Incorporación de nuevos escenarios de uso en el proceso de diseño (SIPD). Propuesta metodológica y valoración de la percepción de los usuarios.” PhD diss., Universitat Jaume I.
Ruiz-Garrido, Miguel F. and Juan Carlos Palmer-Silveira. 2019. “Análisis multimodal del discurso mediante sistemas electrónicos. Un posible avance.” E-Aesla 5, 289-301.
Sánchez-Pérez, María M., ed. 2020. Teacher Training for English-Medium Instruction in Higher Education. Hershey, PA: IGI Global.
Schober, Michael F., David N. Rapp and M. Anne Britt, eds. 2018. Routledge Handbooks in Linguistics: The Routledge Handbook of Discourse Processes. New York and London: Routledge.
Scollon, Ron and Philip LeVine. 2004. “Multimodal Discourse Analysis as the Confluence of Discourse and Technology.” In LeVine and Scollon 2004, 1-6.
Stöckl, Hartmut. 2004. “In between Modes: Language and Image in Printed Media.” In Ventola et al. 2004, 9-30.
Tin, Tan B. 2009. “Features of the most Interesting and the least Interesting Postgraduate Second Language Acquisition Lectures Offered by Three Lecturers.” Language and Education 23 (2), 117-35.
Valeiras-Jurado, Julia and M. Noelia Ruiz-Madrid. 2019. “Multimodal Enactment of Characters in Conference Presentations.” Discourse Studies 27 (5), 561-83.
Ventola, Eija, Cassily Charles and Martin Kaltenbacher, eds. 2004. Perspectives on Multimodality. Amsterdam and Philadelphia: John Benjamins.
Zabalza Beraza, Miguel A. 2011. “Metodología Docente.” Revista de Docencia Universitaria 9 (3), 75-98.
Downloads
Published
How to Cite
Issue
Section
License
The authors retain copyright of articles. They authorise AEDEAN to publish them in its journal Atlantis and to include them in the indexing and abstracting services, academic databases and repositories the journal participates in.
Under the terms of the Creative Commons Attribution NonCommercial ShareAlike 4.0 International Licence (CC BY-NC-SA 4.0), for non-commercial (i.e., personal or academic) purposes only, users are free to share (i.e., copy and redistribute in any medium or format) and adapt (i.e., remix, transform and build upon) articles published in Atlantis, free of charge and without obtaining prior permission from the publisher or the author(s), as long as they give appropriate credit to the author, the journal (Atlantis) and the publisher (AEDEAN), provide the relevant URL link to the original publication and indicate if changes were made. Such attribution may be done in any reasonable manner, but not in any way that suggests the journal endorses the user or their use of the material published therein. Users who adapt (i.e., remix, transform or build upon the material) must distribute their contributions under the same licence as the original.
Self-archiving is also permitted, so that authors are allowed to deposit the published PDF version of their articles in academic and/or institutional repositories, without fee or embargo. Authors may also post their individual articles on their personal websites, again on condition that the original link to the online edition is provided.
Authors are expected to know and heed basic ground rules that preclude simultaneous submission and/or duplicate publication. Prospective contributors to Atlantis commit themselves to the following when they submit a manuscript:
- That no concurrent consideration of the same, or almost identical, work by any other journal and/or publisher is taking place.
- That the potential contribution has not appeared previously, in any form whatsoever, in another journal, electronic format or as a chapter/section of a book.
Seeking permission for the use of copyright material is the responsibility of the author.
Funding data
-
Agencia Estatal de Investigación
Grant numbers PID2021-127827NB-I00
