“This Ticking Noise in My Head”

How Sound Design, Dialogue, Event Structure, and Viewer Working Memory Interact in the Comprehension of Touch of Evil (1958)

in Projections
View More View Less
  • 1 Georgia State University, USA jhutson@gsu.edu
  • 2 Department of Learning Sciences, Georgia State University, USA jmagliano@gsu.edu
  • 3 Department of Psychological Sciences, Birkbeck, University of London, UK tj.smith@bbk.ac.uk
  • 4 Kansas State University, USA loschky@ksu.edu
Restricted access

Abstract

This study tested the role of the audio soundtrack in the opening scene of Orson Welles's Touch of Evil (Orson Welles and Albert Zugsmith, ) in supporting a predictive inference that a time bomb will explode, as the filmmakers intended. We designed two experiments and interpreted their results using the Scene Perception and Event Comprehension Theory (SPECT). Across both experiments, viewers watched the scene, we manipulated their knowledge of the bomb, and they made a predictive inference just before the bomb would explode. Experiment 1 found that the likelihood of predicting the explosion decreased when the soundtrack was absent. Experiment 2 showed that individual differences in working memory accounted for variability in generating the prediction when the soundtrack was absent. We explore the implications for filmmaking in general.

Contributor Notes

John P. Hutson is a Postdoctoral Fellow in Research on the Challenges of Acquiring Language & Literacy at Georgia State University. His research uses an interdisciplinary approach combining cognitive, social, and educational psychology to examine relationships between visual attention and high-level cognitive processes in videos (movies and advertisements), picture stories, and multimedia educational environments. E-mail: jhutson@gsu.edu

Joseph P. Magliano is a professor of Educational Psychology in the Department of Learning Sciences at Georgia State University. He studies the cognitive processes involved in the comprehension of media, such as texts, comics, and films. He uses a variety of empirical approaches in his program of research, such as think aloud protocols, time responses, and event segmentation judgment. He assesses the convergence in these approaches to learn about the nature of comprehension. He also studies what it means to be ready to read for school, and in particular, college and how to support underprepared college readers. E-mail: jmagliano@gsu.edu

Tim J. Smith is Professor of Cognitive Psychology in the Department of Psychological Sciences, Birkbeck, University of London and head of the Cognition in Naturalistic Environments (CINE) Lab (www.cinelabresearch.com). His research covers all aspects of visual cognition with a special focus on the active perception of real and mediated scenes (e.g., TV and Film) via eye movements. E-mail: tj.smith@bbk.ac.uk

Lester Loschky is Professor of Psychological Sciences at Kansas State University, heads the Visual Cognition Laboratory there (www.k-state.edu/psych/vcl). His research areas include: real-world scene perception, human-computer interaction, and perception and comprehension of visual narratives (in film, picture stories, and VR). He studies the interactions between perception, attention, memory, and comprehension processes. To test hypotheses from the Scene Perception & Event Comprehension Theory (SPECT), he investigates how viewers’ comprehension of what they see (their current event model) influences (1) the process of information extraction during each eye fixation, and (2) what viewers selectively attend to through each eye movement. E-mail: loschky@ksu.edu

Projections

The Journal for Movies and Mind

  • Albrecht, Jason E., and Jerome L. Myers. 1998. “Accessing Distant Text Information during Reading: Effects of Contextual Cues.” Discourse Processes 26 (2–3): 87107.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, Richard C., and James W. Pichert. 1978. “Recall of Previously Unrecallable Information Following a Shift in Perspective.” Journal of Verbal Learning and Verbal Behavior 17 (1): 112.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bach, Michael. 2006. “The Freiburg Visual Acuity Test-Variability Unchanged by Post-Hoc Re-Analysis.” Graefe's Archive for Clinical and Experimental Ophthalmology 245 (7): 965971.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Baddeley, Alan. 1992. “Working Memory.” Science 255 (5044): 556559.

  • Batten, Jonathan P., and Tim J. Smith. 2018. “Looking at Sound: Sound Design and the Audiovisual Influences on Gaze.” In Seeing into Screens: Eye Tracking the Moving Image, ed. Tessa Dwyer, Claire Perkins, Sean Redmond and Jodi Sita, 85102. London: Bloomsbury Publishing.

    • Search Google Scholar
    • Export Citation
  • Bläsing, Bettina E. 2015. “Segmentation of Dance Movement: Effects of Expertise, Visual Familiarity, Motor Experience and Music.” Frontiers in Psychology 5: 1500. https://doi.org/10.3389/fpsyg.2014.01500.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bordwell, David. 1985. Narration in the Fiction Film. Madison: University of Wisconsin Press.

  • Bregman, Albert S. 1994. Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge, MA: MIT Press.

  • Brewer, William F., and Keisuke Ohtsuka. 1988. “Story Structure, Characterization, Just World Organization, and Reader Affect in American and Hungarian Short Stories.” Poetics 17 (4–5): 395415.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Calvo, Manuel G. 2001. “Working Memory and Inferences: Evidence from Eye Fixations during Reading.” Memory 9 (46): 365381.

  • Case, Robbie, D. Midian Kurland, and Jill Goldberg. 1982. “Operational Efficiency and the Growth of Short-Term Memory Span.” Journal of Experimental Child Psychology 33 (3): 386404.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chion, Michel. 1994. Audio-Vision: Sound on Screen. Trans. Claudia Gorbman. New York: Columbia University Press.

  • Clark, Herbert H. 1977. “Inferences in Comprehension.” In Basic Processes in Reading: Perception and Comprehension, ed. David LaBerge and S. Jay Samuels, 243263. Hillsdale, NJ: Earlbaum.

    • Search Google Scholar
    • Export Citation
  • Clinton, James A., Stephen W. Briner, Andrew M. Sherrill, Thomas Ackerman, and Joseph P. Magliano. 2017. “The Role of Cinematic Techniques in Understanding Character Affect.” Scientific Study of Literature 7 (2): 177202.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Comito, Terry. 1986. “Welles Labyrinths: Introduction to a Touch of Evil.” Avant Scene Cinema 346–347 (January–February): 634.

    • Search Google Scholar
    • Export Citation
  • Coutrot, Antoine, and Nathalie Guyader. 2016. “Multimodal Saliency Models for Videos.” In From Human Attention to Computational Attention: A Multidisciplinary Approach, ed. Matei Mancas, Vincent P. Ferrera, Nicolas Riche, and John G. Taylor, 291304. New York: Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • D'Angelo, Mike. 2012. “Scenic Routes: Touch of Evil.” The A.V. Club, 19 March 2012. http://www.avclub.com/article/touch-of-evil-71058.

    • Search Google Scholar
    • Export Citation
  • Daneman, Meredyth, and Patricia A. Carpenter. 1980. “Individual Differences in Working Memory and Reading.” Journal of Verbal Learning and Verbal Behavior 19 (4): 450466.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dick, Frederic, Saloni Krishnan, Robert Leech, and Ays¸e Pinar Saygin. 2016. “Environmental Sounds.” In Neurobiology of Language, ed. Gregory Hickok and Steven Small, 11211138. Amsterdam: Elsevier.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gernsbacher, Morton. A. 1990. Language Comprehension as Structure Building, vol. 11. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

  • Gernsbacher, Morton A., Kathleen R. Varner, and Mark E. Faust. 1990. “Investigating Differences in General Comprehension Skill.” Journal of Experimental Psychology: Learning, Memory, and Cognition 16 (3): 430445.

    • Search Google Scholar
    • Export Citation
  • Graesser, Arthur C., and Stanley P. Franklin. 1990. “QUEST: A Cognitive Model of Question Answering.” Discourse Processes 13 (3): 279303.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Graesser, Arthur C., Murray Singer, and Tom Trabasso. 1994. “Constructing Inferences during Narrative Text Comprehension.” Psychological Review 101 (3): 371395.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Henderson, John M., and Andrew Hollingworth. 1999. “High-Level Scene Perception.” Annual Review of Psychology 50: 243271.

  • Huff, M., Tino GK Meitz, and F. Papenmeier. 2014. “Changes in Situation Models Modulate Processes of Event Perception in Audiovisual Narratives.” Journal of Experimental Psychology: Learning Memory and Cognition 40 (5): 13771388.

    • Search Google Scholar
    • Export Citation
  • Hutson, John P., Joseph P. Magliano, and Lester C. Loschky. 2018. “Understanding Moment-to-Moment Processing of Visual Narratives.” Cognitive Science 42: 29993033.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hutson, John P., Tim J. Smith, Joseph P. Magliano, and Lester C. Loschky. 2017. “What Is the Role of the Film Viewer? The Effects of Narrative Comprehension and Viewing Task on Gaze Control in Film.” Cognitive Research: Principles and Implications 2 (1): 130.

    • Search Google Scholar
    • Export Citation
  • Just, Marcel A., and Patricia A. Carpenter. 1992. “A Capacity Theory of Comprehension: Individual Differences in Working Memory.” Psychological Review 99 (1): 122149.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • King, Jonathan, and Marcel A. Just. 1991. “Individual Differences in Syntactic Processing: The Role of Working Memory.” Journal of Memory and Language 30 (5): 580602.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kintsch, Walter. 1988. “The Role of Knowledge in Discourse Comprehension: A Construction-Integration Model.” Psychological Review 95 (2): 163182.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kurby, Christopher A., and Jeffrey M. Zacks. 2008. “Segmentation in the Perception and Memory of Events.” Trends in Cognitive Sciences 12 (2): 7279.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Levin, Daniel T., and Daniel J. Simons. 1997. “Failure to Detect Changes to Attended Objects in Motion Pictures.” Psychonomic Bulletin & Review 4 (4): 501506.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lichtenstein, Edward H., and W. F. Brewer. 1980. “Memory for Goal-Directed Events.” Cognitive Psychology 12 (3): 412445.

  • Liefooghe, Baptist, Pierre Barrouillet, André Vandierendonck, and Valérie Camos. 2008. “Working Memory Costs of Task Switching.” Journal of Experimental Psychology: Learning, Memory, and Cognition 34 (3): 478.

    • Search Google Scholar
    • Export Citation
  • Loschky, Lester C., Adam M. Larson, Joseph P. Magliano, and Tim J. Smith. 2015. “What Would Jaws Do? The Tyranny of Film and the Relationship between Gaze and Higher-Level Narrative Film Comprehension.” PLoS ONE 10 (11): 123.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Loschky, Lester C., Adam M. Larson, Tim J. Smith, and Joseph P. Magliano. 2020. “The Scene Perception & Event Comprehension Theory (SPECT) Applied to Visual Narratives.” Topics in Cognitive Science: 141.

    • Search Google Scholar
    • Export Citation
  • Magliano, Joseph P., Katinka Dijkstra, and Rolf A. Zwaan. 1996. “Generating Predictive Inferences while Viewing a Movie.” Discourse Processes 22 (3): 199224.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Magliano, Joseph P., and Arthur C. Graesser. 1991. “A Three-Pronged Method for Studying Inference Generation in Literary Text.” Poetics 20 (3): 193232.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Magliano, Joseph P., Karyn Higgs, and James Clinton. 2019. “6 Sources of Complexity in Narrative Comprehension across Media.” Narrative Complexity: Cognition, Embodiment, Evolution: 149173.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Magliano, Joseph P., Lester C. Loschky, James A. Clinton, and Adam M. Larson. 2013. “Is Reading the Same as Viewing? An Exploration of the Similarities and Differences Between Processing Text- and Visually Based Narratives.” In Unraveling the Behavioral, Neurobiological, and Genetic Components of Reading Comprehension, ed. Brett Miller, Laurie Cutting and Peggy McCardle, 7890. Baltimore, MD: Brookes Publishing Co.

    • Search Google Scholar
    • Export Citation
  • Magliano, Joseph P., Jason Miller, and Rolf A. Zwaan. 2001. “Indexing Space and Time in Film Understanding.” Applied Cognitive Psychology 15 (5): 533545.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Manfredi, Mirella, Neil Cohn, Mariana De Araújo Andreoli, and Paulo Sergio Boggio. 2018. “Listening beyond Seeing: Event-Related Potentials to Audiovisual Processing in Visual Narrative.” Brain and Language 185: 18.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McKoon, Gail, and Roger Ratcliff. 1998. “Memory-Based Language Processing: Psycholinguistic Research in the 1990s.” Annual Review of Psychology 49 (1): 2542.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McNamara, Danielle S., and Joseph Magliano. 2009. “Toward a Comprehensive Model of Comprehension.” In Psychology of Learning and Motivation, ed. Brian H. Ross, 297384. New York: Elsevier Science.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Meitz, Tino GK, Hauke S. Meyerhoff, and Markus Huff. 2019. “Event Related Message Processing: Perceiving and Remembering Changes in Films with and without Soundtrack.” Media Psychology: 131.

    • Search Google Scholar
    • Export Citation
  • Myers, Jerome L., and Edward J. O'Brien. 1998. “Accessing the Discourse Representation during Reading.” Discourse Processes 26 (2–3): 131157.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Radvansky, Gabriel A. 2012. “Across the Event Horizon.” Current Directions in Psychological Science 21 (4): 269272.

  • Radvansky, Gabriel A., and Jeffrey M. Zacks. 2014. Event Cognition. Oxford: Oxford University Press.

  • Rai, Manpreet K., Lester C. Loschky, and Richard J. Harris. 2015. “The Effects of Stress on Reading: A Comparison of First Language Versus Intermediate Second-Language Reading Comprehension.” Journal of Educational Psychology 107 (2): 348363.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rai, Manpreet K., Lester C. Loschky, Richard J. Harris, Nicole R. Peck, and Lindsay G. Cook. 2011. “Effects of Stress and Working Memory Capacity on Foreign Language Readers’ Inferential Processing during Comprehension.” Language Learning 61 (1): 187218.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Reynolds, Jeremy R., Jeffrey M. Zacks, and Todd S. Braver. 2007. “A Computational Model of Event Segmentation from Perceptual Prediction.” Cognitive Science 31 (4): 613643.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sargent, Jesse Q., Jeffrey M. Zacks, David Z. Hambrick, Rose T. Zacks, Christopher A. Kurby, Heather R. Bailey, Michelle L. Eisenberg, and Taylor M. Beck. 2013. “Event Segmentation Ability Uniquely Predicts Event Memory.” Cognition 129 (2): 241255.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, Murray. 1995. “Film Spectatorship and the Institution of Fiction.” Journal of Aesthetics and Art Criticism 53 (2): 113127.

  • Smith, Tim. J. 2014. “Audiovisual Correspondences in Sergei Eisenstein's Alexander Nevsky: A Case Study in Viewer Attention.” In Cognitive Media Theory (AFI Film Reader), ed. P. Taberham and T. Nannicelli. Abingdon, UK: Taylor & Francis.

    • Search Google Scholar
    • Export Citation
  • Smith, Tim J., and John M. Henderson. 2008. “Edit Blindness: The Relationship between Attention and Global Change Blindness in Dynamic Scenes.” Journal of Eye Movement Research 2 (2:6): 117.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, Tim J., and Janet Yvonne Martin-Portugues Santacreu. 2017. “Match-Action: The Role of Motion and Audio in Creating Global Change Blindness in Film.” Media Psychology 20 (2): 317348.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • St. George, Marie, Suzanne Mannes, and James E. Hoffman. 1997. “Individual Differences in Inference Generation: An ERP Analysis.” Journal of Cognitive Neuroscience 9 (6): 776787.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Truffaut, François, Alfred Hitchcock, and Helen G. Scott. 1984. Hitchcock, revised edition. New York: Simon & Schuster.

  • Tully, Tim. 1999. “The Sounds of Evil.” Filmsound. http://filmsound.org/murch/evil/ (accessed 27 February 2020).

  • Turner, Marilyn L., and Randall W. Engle. 1989. “Is Working Memory Capacity Task Dependent?Journal of Memory and Language 28 (2): 127154.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Van den Broek, Paul. 1990. “The Causal Inference Maker: Towards a Process Model of Inference Generation in Text Comprehension.” Comprehension Processes in Reading: 423445.

    • Search Google Scholar
    • Export Citation
  • Welles, Orson, and Rick Schmidlin. 1998. Touch of Evil [Film]. Universal Studios.

  • Welles, Orson, and Albert Zugsmith. 1958. Touch of Evil [Film]. Universal Pictures.

  • Whitney, Paul, Bill G. Ritchie, and Matthew B. Clark. 1991. “Working-Memory Capacity and the Use of Elaborative Inferences in Text Comprehension.” Discourse Processes 14 (2): 133145.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Yarbus, Alfred L. 1967. Eye Movements and Vision.Trans. Basil Haigh. New York: Plenum Press.

  • Zacks, Jeffrey M. 1999. Event Structure Perception Studies in Perceiving, Remembering, and Communicating. Stanford, CA: Stanford University.

    • Search Google Scholar
    • Export Citation
  • Zacks, Jeffrey M. 2004. “Using Movement and Intentions to Understand Simple Events.” Cognitive Science 28 (6): 9791008.

  • Zacks, Jeffrey M., Todd S. Braver, Margaret A. Sheridan, David I. Donaldson, Abraham Z. Snyder, John M. Ollinger, Randy L. Buckner, and Marcus E. Raichle. 2001. “Human Brain Activity Time-Locked to Perceptual Event Boundaries.” Nature Neuroscience 4 (6): 651655.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zacks, Jeffrey M., Shawn Kumar, Richard A. Abrams, and Ritesh Mehta. 2009. “Using Movement and Intentions to Understand Human Activity.” Cognition 112 (2): 201216.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zacks, Jeffrey M., Nicole K. Speer, and Jeremy R. Reynolds. 2009. “Segmentation in Reading and Film Comprehension.” Journal of Experimental Psychology: General 138 (2): 307327.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zacks, Jeffrey M., and Khena M. Swallow. 2007. “Event Segmentation.” Current Directions in Psychological Science 16 (2): 8084.

  • Zwaan, Rolf A., and Gabriel A. Radvansky. 1998. “Situation Models in Language Comprehension And Memory.” Psychological Bulletin 123 (2): 162185.

    • Crossref
    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 517 517 47
Full Text Views 71 71 2
PDF Downloads 66 66 3