Sound Anchors

A Cognitive and Multimodal Approach to Sound and Narrative Structure in Film

in Projections
Author:
Brad Jackson Doctoral Candidate, The University of British Columbia, Canada bradley.jackson@alumni.ubc.ca

Search for other papers by Brad Jackson in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

When watching a film, we engage with much more than combinations of moving images. We combine what we see with what we hear, and what we hear often aids in the construction of a story. Although some researchers endorse the ways sound guides viewer expectations, there is still a need to explain the ways images, sounds, and other available cinematic modes interact to construct meaning. This article engages with research on embodiment, cognition, and multimodal artifacts to reveal how sound aids in the construction of film narratives by focusing on examples where sounds take the primary role in constructions of narrative meaning. Additionally, by discussing recent theories on cognition and multimodality, this article shows how sounds can evoke conceptual and narrative information in ways that stabilize our understanding of cinematic representations through the joint contribution of all of the available modes.

Contributor Notes

Brad Jackson is a doctoral candidate at The University of British Columbia. His doctoral research focuses on the intersections of cognitive science, cognitive linguistics, and narrative film, specifically, the role of multimodality in film analysis. Email: bradley.jackson@alumni.ubc.ca

  • Collapse
  • Expand

Projections

The Journal for Movies and Mind

  • Altman, Rick. 2017. “The Early Cinema Soundscape.” In The Routledge Companion to Screen Music and Sound, ed. Miguel Mera, Ronald Sadoff, and Ben Winters, 190200. New York: Routledge. https://doi.org/10.4324/9781315681047-16.

    • Search Google Scholar
    • Export Citation
  • Bateman, John A., and Karl-Heinrich Schmidt. 2012. Multimodal Film Analysis. New York: Routledge. https://doi.org/10.4324/9780203128220.

    • Search Google Scholar
    • Export Citation
  • Bateman, John A., Janina Wildfeuer, and Tuomo Hiippala. 2017. Multimodality: Foundations, Research and Analysis a Problem-Oriented Introduction. Boston: De Gruyter/Mouton. https://doi.org/10.1515/9783110479898.

    • Search Google Scholar
    • Export Citation
  • Bordwell, David. (1985) 1988. “The Introduction of Sound.” In The Classical Hollywood Cinema: Film Style & Mode of Production to 1960, ed. David Bordwell, Janet Staiger, and Kristin Thompson, 536–547. New York: Columbia University Press. https://doi.org/10.4324/9780203358818.

    • Search Google Scholar
    • Export Citation
  • Bordwell, David, Kristin Thompson, and Jeff Smith. 2020. Film Art: An Introduction. 12th edn. New York: McGraw-Hill Higher Education.

  • Chion, Michel. (1990) 1994. Audio-Vision: Sound on Screen, ed. and trans. Claudia Gorbman. New York: Columbia University Press.

  • Coëgnarts, Maarten. 2019. Film as Embodied Art: Bodily Meaning in the Cinema of Stanley Kubrick. Brookline, MA: Academic Studies Press.

    • Search Google Scholar
    • Export Citation
  • Coëgnarts, Maarten, and Peter Kravanja. 2015. “Embodied Cinematic Subjectivity: Metaphorical and Metonymical Modes of Character Perception in Film.” In Embodied Cognition and Cinema, ed. Maarteen Coëgnarts and Peter Kravanja, 221243. Leuven: Leuven University Press.

    • Search Google Scholar
    • Export Citation
  • Cohen, Annabel J.. 2013. “Film Music from the Perspective of Cognitive Science.” In The Oxford Handbook of Film Music Studies, ed. David Neumeyer, 137. Oxford: Oxford University Press.

    • Search Google Scholar
    • Export Citation
  • Dancygier, Barbara. 2012. The Language of Stories: A Cognitive Approach. Cambridge: Cambridge University Press. https://doi:10.1017/CBO9780511794414.

    • Search Google Scholar
    • Export Citation
  • Fauconnier, Gilles. 1997. Mappings in Thought and Language. New York: Cambridge University Press. https://doi.org/10.1017/CBO9781139174220.

    • Search Google Scholar
    • Export Citation
  • Fauconnier, Gilles. 2003. “Conceptual Blending, Form and Meaning.” Recherches En Communication 19: 5786. https://doi.org/10.14428/rec.v19i19.48413.

    • Search Google Scholar
    • Export Citation
  • Fauconnier, Gilles, and Mark Turner. 1998. “Conceptual Integration Networks.” Cognitive Science 22 (2): 133187. https://doi.org/10.1207/s15516709cog2202_1.

    • Search Google Scholar
    • Export Citation
  • Fauconnier, Gilles, and Mark Turner. (2002) 2008. The Way We Think: Conceptual Blending and the Mind's Hidden Complexities. New York: Basic Books.

    • Search Google Scholar
    • Export Citation
  • Fillmore, Charles. 1982. “Frame Semantics.” In Linguistics in the Morning Calm, ed. The Linguistic Society of Korea, 111137. Seoul: Hanshin Publishing Co.

    • Search Google Scholar
    • Export Citation
  • Genette, Gérard. 1980. Narrative Discourse: An Essay in Method. New York: Cornell University Press.

  • Gordejuela, Adriana. 2021. Flashbacks in Film: A Cognitive and Multimodal Analysis. New York: Routledge. https://doi.org/10.4324/9781003153573.

    • Search Google Scholar
    • Export Citation
  • Hutchins, Edwin. 2005. “Material Anchors for Conceptual Blends.” Journal of Pragmatics 37 (10): 15551577. https://doi.org/10.1016/j.pragma.2004.06.008.

    • Search Google Scholar
    • Export Citation
  • Müller, Cornelia, and Hermann Kappelhoff. 2018. Cinematic Metaphor: Experience—Affectivity—Temporality. Boston: De Gruyter.

  • Oakley, Todd, and Vera Tobin. 2012. “Attention, Blending, and Suspense in Classic and Experimental Film.” In Blending and the Study of Narrative, ed. Ralph Schneider and Marcus Hartner, 5783. Berlin: Mouton de Gruyter.

    • Search Google Scholar
    • Export Citation
  • O'Halloran, Kay L. 2004. Multimodal Discourse Analysis: Systemic-Functional Perspectives. New York: Continuum.

  • Tseng, Chiao-I, and John A. Bateman. 2012. “Multimodal Narrative Construction in Christopher Nolan's Memento: A Description of Analytic Method.” Visual Communication 11 (1): 91119. https://doi.org/10.1177/1470357211424691.

    • Search Google Scholar
    • Export Citation
  • Tseng, Chiao-I, Jochen Laubrock, and John A. Bateman. 2021. “The Impact of Multimodal Cohesion on Attention and Interpretation in Film.” Discourse, Context & Media 44: 100544. https://doi.org/10.1016/j.dcm.2021.100544.

    • Search Google Scholar
    • Export Citation
  • Wildfeuer, Janina. 2014a. “Coherence in Film: Analysing the Logical Form of Multimodal Narrative Discourse.” In Multimodal Epistemologies: Towards an Integrated Framework, ed. Arianna Maiorani and Christine Christie, 260274. New York: Routledge.

    • Search Google Scholar
    • Export Citation
  • Wildfeuer, Janina. 2014b. Film Discourse Interpretation: Towards a New Paradigm for Multimodal Film Analysis. New York: Routledge. https://doi.org/10.4324/978020376620.

    • Search Google Scholar
    • Export Citation
  • Wingstedt, Johnny. 2005. “Narrative Music: Towards an Understanding of Musical Narrative Functions in Multimedia.” PhD diss., Luleå tekniska universitet, Musikhögskolan i Pitea.

  • Wingstedt, Johnny. 2008. “Making Music Mean: On Functions of and Knowledge about Narrative Music in Multimedia.” PhD diss., Luleå University of Technology.

    • Search Google Scholar
    • Export Citation
  • Wingstedt, Johnny, Sture Brändström, and Jan Berg. 2010. “Narrative Music, Visuals and Meaning in Film.” Visual Communication 9 (2): 193210. https://doi.org/10.1177/1470357210369886.

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 1183 553 92
Full Text Views 148 38 3
PDF Downloads 199 84 3