“Mind the Gap”

Between Movies and Mind, Affective Neuroscience, and the Philosophy of Film

in Projections

Abstract

Murray Smith’s Film, Art, and the Third Culture makes a significant contribution to cognitive film theory and philosophical aesthetics, expanding the conceptual tools of film analysis to include perspectives from neuroscience and evolutionary psychology. Smith probes assumptions about how cinema affects spectators by examining aspects of experience and neurophysiological responses that are unavailable to conscious, systematic reflection. This article interrogates Smith’s account of emotion, empathy, and imagination in cinematic representation and film spectatorship, placing his work in dialogue with other recent interventions in the fields of cinema studies and embodied cognition. Smith’s contribution to understanding the role of emotion in screen studies is vital, and when read in conjunction with recent publications by Carl Plantinga and Mark Johnson on ethical engagement and the moral imagination, this new work constitutes a notable advance in film theory.

The final page of Murray Smith’s Film, Art, and the Third Culture concludes with a discussion of the gap between the mind, mental representations, mediated representations such as film footage, and the world that such representations depict: “Mind the gap,” Smith quips, borrowing the words of philosopher Jerry Fodor (2017: 223). Throughout the book, Smith turns his mind to this gap and to the gap between the arts and the sciences, arguing that imaginative, internal representational capacities of the mind and visual prostheses such as cameras that technologically mediate external representations are tools that enable us to grasp and make sense of the world. Smith’s thought-provoking book establishes an integrated research approach that he refers to as a naturalized aesthetics of film, which sees cinema as a technocultural product of fundamental human capacities related to perception, cognition, and emotion. Taking a biocultural approach that examines the interrelationship between biological capacities and their cultural elaboration, he demonstrates that both scientific and humanistic perspectives have roles to play in illuminating artistic creativity and aesthetic experience (2017: 10).

Film, Art, and the Third Culture: A Naturalized Aesthetics of Film joins Carl Plantinga’s Screen Stories: Emotion and the Ethics of Engagement (2018) and Mark Johnson’s The Aesthetics of Meaning and Thought: The Bodily Roots of Philosophy, Science, Morality, and Art (2018) in the latest wave of cognitivist research concerned with film and art. Such research demonstrates that neuroscientific measures and imaging techniques offer new forms of insight into the audience’s experience of film that extend and supplement traditional methods of critical reflection, reception studies, and film analysis.

This article focuses on Part 2 of Smith’s book, “Science and Sentiment,” which builds on his earlier work, Engaging Characters (1995), to study emotional engagement with cinema and to analyze the representation and expression of emotion in film. My particular interest is in the account Smith advances of the nature of emotion and its role in relation to empathy and imagination in the film experience. As Smith contends, quantitative empirical research adds scientific evidence and methods to the qualitative, experiential evidence marshalled by the filmgoing public and humanities researchers. Neuroscientific studies can provide evidence to support or critique conceptual assumptions about cinema spectatorship and empathy by using medical imaging techniques and neurophysiological measures to transcend “the limits of ordinary human perception” in order to examine aspects of empathy and film that are unavailable to conscious experience, reflection, or textual analysis (Smith 2017: 103). Rather than succumbing to the idea that neuroscientific methods can demystify the mind and thus explain aesthetic experience and responses to film, Smith argues for “triangulating” or cross-checking the phenomenological experience of film with psychological and neural evidence by using three interrelated levels of analysis to understand mental phenomena (2017: 11). In particular, in his chapters “Triangulating Aesthetic Experience” and “The Engine of Reason and the Pit of Naturalism,” Smith argues persuasively that “neural evidence sheds light on the functional nuances of the phenomena that elude ordinary experience and reflection” (2017: 103). Starting with emotion and moving on to empathy and then imagination, I will map out and critically engage with Smith’s naturalized aesthetics of film.

Emotion

In a book that delights in abstruse titles and subheadings such as “Feeling Prufish” and “Normative Panting,” it is refreshing to find systematic explanations and clear definitions of complex concepts and processes. Smith presents a compelling argument that emotions have evolved through natural selection to provide humans with an evolutionary advantage. Recognition of emotional facial expressions, he writes, provides “reliable information about the inner states of others” and this facilitates imaginative predictions of behavior within social contexts or narrative scenarios (2017: 142). Emotions can thus play a motivational role in guiding actions and responses to others, and they can function as intensifiers or “signal boosters” in the processes of decision-making (130).

From Smith’s biocultural perspective, an emotion is an embodied appraisal, that is, “a dynamic somatic and cognitive apprehension of the significance of some phenomena – an object, a person, an event, a situation – by an agent” (162). By contrast, Carl Plantinga uses the related but somewhat more awkward term concern-based construals to indicate that emotions can be understood as components of cognitive activities such as critique, inference, and reconsideration. “Emotions are intentional states, but they are embodied and thus accompanied by physiological disturbances, feelings, and action tendencies. Emotions may result from prior thought and perception, and they result in subsequent thought and perception” (2018: 113), Plantinga writes. For Smith and Plantinga alike, the embodied, felt nature of emotion is central to its role in cognition and deliberation. As Charles Darwin points out in The Expression of the Emotions in Man and Animals, “most of our emotions are so closely connected with their expression, that they hardly exist if the body remains passive”; for instance, a person may be consumed by angry thoughts, “but until his bodily frame is affected he cannot be said to be enraged” (qtd in Smith 2017: 134). The cinematic display of emotion, magnified on the big screen, thereby serves important cultural functions. Smith sees cultural artforms such as narrative cinema as coaching audiences in the interpretation of emotion and schooling us in the forms of imaginative engagement and emotional literacy upon which altruistic interaction and social coherence rely. Fostering empathy is an important aspect of this function.

Empathy

While empathy is not an emotion in its own right, it serves as a form of affective mapping, just as emotions furnish us with a map of values, valences, harms, and benefits (Smith 2017: 179). Smith defines empathy as “a term which is used to refer to a variety of phenomena, ranging from the conscious, imaginative effort to ‘perspective take’ or put oneself in another’s shoes, to affective mimicry and emotional contagion, whereby we ‘catch’ the emotions of others through a process of low-level, non-conscious, involuntary mimicry” (2017: 99). He presents an account of the role of empathy in character engagement from a cognitivist-philosophical perspective, taking into account recent research about neural mirroring and empathic simulation in neuroscience and drawing distinctions between intentional, cognitive mindreading and involuntary, neurobiological processes that he refers to as mindfeeling (194). Both capacities are integral to the social nature of human life and to the way we make sense of the world (99). Indeed, as Plantinga puts it, cognition itself is not limited to mental activity; it is “suffused with affect” (2018: 113).

As an audiovisual medium of narrative communication, film both represents and elicits the emotional and empathic exchanges that are central to human relationships and to ethical understanding. The stylistic and technological audiovisual strategies by which film creates an aesthetic experience for its audience can also reproduce a screen protagonist’s perceptual experience, thereby stimulating both the imaginative and affective components of empathy. Neural mirroring or mimicry of another person’s emotional state via facial feedback or by experiencing emotional contagion may not properly be considered empathy in isolation because such involuntary affective states are experienced as one’s own feelings rather than being intentional and other-oriented. However, Smith argues convincingly that these affective processes interrelate and that “mindfeeling” can extend the ability to attune ourselves to others and grasp their emotional states as we then consciously engage in the process of what he terms personal or central imagining (2017: 101). As Smith explains, neurobiological mirroring processes can “‘wire’ us into each other,” enabling a “direct, experiential form of understanding” of other people’s (or film characters’) actions and emotions (100). In terms of the aforementioned triangulation of perspectives, the mirror neuron system “is the neural substrate that ‘implements’ our psychological capacities for sensory, motor, and affective mimicry, and the experiences that may characterize these processes” (99).

Thus, by Smith’s account, emotions help us to understand ourselves and our world and empathic mirroring of others’ emotions constitutes “direct experiential” knowledge through which we can feel in our own bodies how another person feels (180). Feeling, knowing, and imagining are clearly not the same as each other, but each provide pathways to understanding. As Smith goes on to explain, involuntary bodily responses such as affective mimicry and emotional contagion can prompt and support volitional cognitive acts of empathic simulation and imaginative activity: “Mimicry of basic actions and emotions may scaffold the imagination, including the empathic imagination, of more elaborate, finely specified states of mind” (180). Detailing the interrelationship between feeling and thought in complex phenomena like the empathic imagination is one of Smith’s greatest contributions to film theory.

Imagination

Dating back to his work in Engaging Characters, Smith has characterized empathy as a type of imagining that he refers to as personal imagining or central imagining. Central imagining involves mentally projecting a possible scenario and imagining, perceiving, or experiencing that scenario in a way that enables a better understanding of those with whom we empathize. Importantly, Smith characterizes empathic imagining as “other-focussed personal imagining – imagining the experiences of others ‘from the inside,’” stating that such imagining allows us to identify, understand, and directly apprehend the emotional moods and mindsets of others (2017: 179). However, unlike empathy, Smith does not define imagination with reference to philosophical or scientific research. Instead, he considers the evolutionary advantage of cultural forms such as cinema that both spring from and give rise to imaginative activity.

An evolutionary advantage of fiction and narrative film may be that they hone the capacity to imagine: “One thing that sets us apart from other species is our ability to ‘simulate’, in our minds, circumstances which we might encounter, or indeed which we have encountered in the past” (129). Using an uncharacteristically flamboyant analogy, Smith develops the notion that the way cognitive and reproductive systems work is based on evolutionary adaptations that advance the species, suggesting that “fiction is to the imagination as strawberry cheesecake is to fruit – a cultural elaboration of a biological adaptation” (129). This suggests that imagination enriches foresight and the ability to plan, which in turn improves the chances of successful interaction with others and the environment. Furthermore, artistic representations of empathy scenarios, such as those found in narrative films, provide elaborate examples of imagining that may augment the reach, scale, and intensity of empathy (190–191). In a compelling passage about the power of film, Smith writes:

Our ability to empathise is extended across a wide range of types of person and situation, and sustained and intensified by virtue of the artificial, “designed” environments created by narrative artefacts … The possibility of understanding “from the inside” – that is, empathically imagining the thoughts and feelings of – human agents in social situations more or less radically different from our own is disclosed. We may come not only to see, but to feel, how an agent in a given situation concludes that there are only a particular set of viable choices open to them.

(191–192)
This gets to the heart of why emotional and empathic engagement with cinema and other arts is significant and worth studying from both scientific and humanistic perspectives. It is a topic that philosopher Mark Johnson takes up in relation to ethics, imagination, and the neural underpinnings of embodied cognition:

Moral imagination – both as our capacity to empathically understand and feel with others and our ability to imagine how experience would play out under the shaping influence of various values and choices – is thus dependent on our ability to simulate experiences. The new approach known as “simulation semantics” has begun to explore experimentally how people understand experiences, visual scenes, and linguistic expressions by simulating, in their own neural and bodily systems, the perceptual experiences, motor programs, and affective valences that are involved in someone having those experiences. According to this perspective, conceptualization and reasoning are experiential simulations.

(2018: 172)
I have quoted Johnson’s work at length because of the central relevance of his research to that of both Smith and Plantinga, but also because with the increasing prominence of immersive technologies such as virtual reality I believe his assessment of the role of simulation in moral imagination is the next problem that cognitive media studies and those of us working at the intersection of film and philosophy need to tackle. Alongside Smith’s and Plantinga’s work, Johnson’s model of ethical deliberation as a process of imaginative projection that works through “cognitive-conative-affective simulation” (2018: 173) may provide the key to solving this problem.

Film as an Extension of Mind

In his chapter “Empathy, Expansionism and the Extended Mind,” Smith uses the theory of the “extended mind,” or the idea that cognition is not just located in the brain but is embodied and extends into the environment, to elucidate how empathy works in film narratives. Here, Johnson’s influential account of how meaning arises from physical bodily engagement with the environment in Embodied Mind, Meaning, and Reason (2017) is pertinent. A former student of Paul Ricoeur, Johnson draws together insights from the phenomenological and hermeneutic traditions with those of cognitive science and the philosophy of art. In The Aesthetics of Meaning and Thought (2018), Johnson’s account of the empathic imagination and the moral imagination in relation to art and narrative fiction is particularly relevant to Smith’s project. Johnson argues that the moral imagination depends on our ability to simulate experiences and that this ability is cultivated to a large extent through engagement with art (2018: 172). Building on the work of Martha Nussbaum (1990), among others, Johnson shows that “moral sensitivity requires us to be able to imagine the lives of others, and that narrative fictions are one of the primary vehicles for such a process of empathic imagination” (2018: 166). The question remains how the imagination works with and through the technologically mediated experience of narrative cinema to foster empathic insight. It is a small step from here to considering the ways in which embodied engagement with the technologies and narratives of an artform such as cinema could constitute what Smith considers to be a form of extended mind.

Extended mind theory proposes that “some part of the world is reliably coupled with the mind to form an integrated cognitive system; it is in this sense that the mind is extended into the world, structuring and co-opting part of it in order to augment its processing capabilities” (Smith 2017: 185). More particularly, Smith states that “when we empathize with another person we extend our mind to couple with part of her mind” and thereby learn something about her subjective circumstances by “co-opting” her perceptual and emotive capacities (188). But based on what we know about mirror neurons, embodied simulation entails an involuntary process of intercorporeality rather than extending one’s own mind to “co-opt” or “couple” with another person’s mind. Intercorporeality, according to Vittorio Gallese, is a kind of social intelligence derived from the observation and neural mirroring of bodily actions, gestures, and expressions that enable affective, empathic understanding of other subjects’ agency and behavioral intentions (2016: 302). Cinematic narratives with emotive close-ups and images of the human body in action and interaction would therefore seem to provide an ideal laboratory for intercorporeal simulation.

Smith thinks of filmmaking technologies such as cinematography and CGI as “cognitive prostheses,” just as technologies such as telescopes and microscopes act as “perceptual prostheses – devices that amplify our native perceptual capacities” (2017: 188). In this way, practices of representation and narration from childhood games of mimicry and make believe through to cinematic technologies and virtual reality simulations can augment empathy by reinforcing and extending the imagination (188–189). Specifically, Smith proposes that empathy can be enhanced by certain cognitive prostheses or mental extensions, and in such cases “it is the domain of representation, and especially the practice of narration, that constitutes the ‘environmental support’ created by the mind to drive its amplified performance” (188). If I understand this correctly, then it may not be so much the case that when crafting or responding to film narratives the mind creates an environmental support that boosts its performance, but more simply that cinematic technologies can amplify perceptual attunement to others. With reference to the work of Patrick Maynard, Smith argues that cinematographic technology is a “cognitive amplifier” that expands the capacity for visualization and imagination (187). In other words, the technologically mediated representations of narrative cinema can focus and augment empathic attention via techniques such as visual and aural close-ups and the position-taking cadence of shot-reverse-shot editing.

Compare this to phenomenologist Vivian Sobchack’s account of hermeneutic and embodiment relations with technology in Address of the Eye (1992: 186–195), where she discusses different levels on which cinematic technology informs and extends perception and cognition. Sobchack details the way in which the cinema audience perceives through technology (as one sees through a clear lens) and perceives along with the technology (intersubjectively perceiving what the camera “sees,” what the microphone “hears,” and what the projector “expresses,” as though these are the organs of the film’s own embodied subjectivity). In this account, the film along with its filmmakers and its audience can thus be conceptualized as an intersubjective entity that is more like a cyborg than a prosthetic device that technologically extends human perception (1992: 163). In a passage that calls to mind Smith’s suggestion that film is a technological extension of mind that integrates with and augments our cognitive system, Sobchack argues that film allows “for the extension of human intentionality” such that “the machine is incorporated into the human intentional act of perceiving the world, even as the machine enables a patently ‘impossible’ human perception, that is, one otherwise unrealizable without the machine’s incorporation” (Sobchack 1992: 184).

Just as Sobchack is persuasive in her attention to different levels of technologically mediated phenomenological experience, Smith is persuasive in his call for taking the insights of cognitive science seriously. It is worth noting that Smith does not restrict his extended mind thesis to cinematic technologies and narratives; he also describes neuroscientific tools as technologies that enable a form of “scientific observation that transcends the limits of ordinary human perception; the brain scanner joins the telescope, the microscope, stop motion and x-ray, radar and sonar, and so on – all technologies which allow us to see (or otherwise detect) new aspects of our world, or to see familiar aspects of it in greater detail” (2017: 103).

To conclude, neither the concept of the extended mind nor that of the film’s body provides a completely convincing explanation of the film audience’s embodied and imaginative extensions through cinematic technologies and narratives into other worlds in ways that forge empathic connections to others, but both perspectives open onto important avenues for further research. This means more genuinely interdisciplinary collaborative work is needed to understand how imagination and empathy work and how they may be affected by new technologies and aesthetic techniques. While it remains true that “scientific practice will never replace art as a vehicle of human expression and meaning-making; and neither will science provide the tools for discerning the ‘embodied meanings’ of artworks” (Smith 2017: 220), Smith’s book brings together some of the best scholarship from both paradigms to help us consider aesthetic experience anew.

References

  • GalleseVittorio. 2016. “Finding the Body in the Brain: From Simulation Theory to Embodied Simulation.” In Goldman and His Critics ed. Brian P. McLaughlin and Hilary Kornblith297317. Hoboken, NJ: John Wiley and Sons.

    • Search Google Scholar
    • Export Citation
  • JohnsonMark. 2017. Embodied Mind Meaning and Reason: How Our Bodies Give Rise to Understanding. Chicago: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • JohnsonMark. 2018. The Aesthetics of Meaning and Thought: The Bodily Roots of Philosophy Science Morality and Art. Chicago: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • NussbaumMartha. 1990. Love’s Knowledge: Essays on Philosophy and Literature. New York: Oxford University Press.

  • PlantingaCarl. 2018. Screen Stories: Emotion and the Ethics of Engagement. Oxford: Oxford University Press.

  • SmithMurray. 1995. Engaging Characters: Fiction Emotion and the Cinema. Oxford: Clarendon Press.

  • SmithMurray. 2017. Film Art and the Third Culture: A Naturalized Aesthetics of Film. Oxford: Oxford University Press.

  • SobchackVivian. 1992. Address of the Eye: A Phenomenology of Film Experience. Princeton, NJ: Princeton University Press.

If the inline PDF is not rendering correctly, you can download the PDF file here.

Contributor Notes

Jane Stadler is Professor of Film and Media Studies and Chair of the Department of Media and Communication at Swinburne University, Australia. She led a collaborative Australian Research Council research project about landscape and location in Australian cinema, literature, and theater (2011–2014) and coauthored Imagined Landscapes: Geovisualizing Australian Spatial Narratives (2016). She is the author of Pulling Focus: Intersubjective Experience, Narrative Film and Ethics (2008) and coauthor of Screen Media (2009) and Media and Society (2016). Her current research interests include visceral cinema, phenomenological and philosophical approaches to screen aesthetics, ethics, imagination and spectatorship, and neurocinematic studies of audience responses. Email: jstadler@swin.edu.au

Projections

The Journal for Movies and Mind

  • GalleseVittorio. 2016. “Finding the Body in the Brain: From Simulation Theory to Embodied Simulation.” In Goldman and His Critics ed. Brian P. McLaughlin and Hilary Kornblith297317. Hoboken, NJ: John Wiley and Sons.

    • Search Google Scholar
    • Export Citation
  • JohnsonMark. 2017. Embodied Mind Meaning and Reason: How Our Bodies Give Rise to Understanding. Chicago: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • JohnsonMark. 2018. The Aesthetics of Meaning and Thought: The Bodily Roots of Philosophy Science Morality and Art. Chicago: University of Chicago Press.

    • Search Google Scholar
    • Export Citation
  • NussbaumMartha. 1990. Love’s Knowledge: Essays on Philosophy and Literature. New York: Oxford University Press.

  • PlantingaCarl. 2018. Screen Stories: Emotion and the Ethics of Engagement. Oxford: Oxford University Press.

  • SmithMurray. 1995. Engaging Characters: Fiction Emotion and the Cinema. Oxford: Clarendon Press.

  • SmithMurray. 2017. Film Art and the Third Culture: A Naturalized Aesthetics of Film. Oxford: Oxford University Press.

  • SobchackVivian. 1992. Address of the Eye: A Phenomenology of Film Experience. Princeton, NJ: Princeton University Press.