On-Beat/Off-Beat

Visual Responses to Audio-Visual Asynchrony in Music Videos

in Projections
Author:
Thorbjörn Swenberg Senior Lecturer, Moving Image Production Department, Dalarna University, Sweden. tsw@du.se

Search for other papers by Thorbjörn Swenberg in
Current site
Google Scholar
PubMed
Close
and
Simon Carlgren simoncarlgren@gmail.com

Search for other papers by Simon Carlgren in
Current site
Google Scholar
PubMed
Close
Restricted access

Abstract

Audio-visual rhythm can be achieved in a variety of ways, in film as well as in music videos. Here, we have studied human visual responses to video editing with regard to musical beats, in order to better understand the role of visual rhythm in an audio-visual flow. While some suggest that music videos should maintain synchrony in the audio-visual rhythm, and others claim that music videos should be rhythmically loose in their structure, there is a functional aspect of vision and hearing that reacts to the juxtaposition of audio and visual rhythms. We present empirical evidence of cognitive effects, as well as perceptual differences with attentional effects, for viewers watching music videos cut on-beat and off-beat.

Contributor Notes

Thorbjörn Swenberg has a PhD in Innovation and Design and is a Senior Lecturer at the Moving Image Production Department at Dalarna University, Sweden. His research interests include creative decision-making in media production, design of audiovisual objects, and viewer evaluation of audiovisual assemblies. His thesis explores these topics through a qualitative interpretation of design practice combined with quantitative analyses of eye movement data gathered in a set of experiments. E-mail: tsw@du.se

Simon Carlgren has a BA in Audiovisual Production, a MA in Film Editing, and is currently working as a documentary film editor. E-mail: simoncarlgren@gmail.com

  • Collapse
  • Expand

Projections

The Journal for Movies and Mind

  • Alais, David, Fiona Newell, and Pascal Mamassian. 2010. “Multisensory Processing in Review: From Physiology to Behaviour.” Seeing and Perceiving 23 (1): 338. https://doi.org/10.1163/187847510X488603.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Anderson, Joseph D., and Barbara F. Anderson, eds. 2005. Moving Image Theory: Ecological Considerations. Carbondale: Southern Illinois University Press.

    • Search Google Scholar
    • Export Citation
  • Arndt, Petra A., and Hans Colonius. 2003. “Two Stages in Crossmodal Saccadic Integration: Evidence from a Visual-Auditory Focused Attention Task.” Experimental Brain Research 150 (4): 417426. https://doi.org/10.1007/s00221-003-1424-6

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Batten, Jonathan. P., and Tim J. Smith. 2018. “Saccades Predict and Synchronize to Visual Rhythms Irrespective of Musical Beats.” Visual Cognition 26 (9): 695718. doi.org/10.1080/13506285.2018.1544181.

    • Crossref
    • Export Citation
  • Beatty, Jackson. 1982. “Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources.” Psychological Bulletin 91: 276292. https://doi.org/10.1037/0033-2909.91.2.276.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Benedetto, Alessandro, David C. Burr, and Maria C. Morrone. 2018. “Perceptual Oscillation of Audiovisual Time Simultaneity.” eneuro 5 (3). https://doi.org/10.1523/ENEURO.0047-18.2018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Benedetto, Alessandro, Maria C. Morrone, and Alice Tomassini. 2020. “The Common Rhythm of Action and Perception.” Journal of Cognitive Neuroscience 32 (2): 187200. https://doi.org/10.1162/jocn_a_01436.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Björnberg, A. 1990. “Sign of the Times? Om Musikvideo och Populärmusikens Semiotik” [Music-video and the semiotics of popular music]. Svensk Tidskrift för Musikforskning 72: 6384. http://www.musikforskning.se/stm/STM1990/STM1990Bjornberg.pdf.

    • Search Google Scholar
    • Export Citation
  • Blair, Christopher D., and Jelena Ristic. 2019. “Attention Combines Similarly in Covert and Overt Conditions.” Vision 3 (2): 16. https://doi.org/10.3390/vision3020016.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Block, Bruce A. 2001. The Visual Story: Seeing the Structure of Film, TV, and New Media. London: Focal Press.

  • Burr, David. 2005. “Vision: In the Blink of an Eye.” Current Biology 15 (14): R554R556.

  • Buswell, Guy. 1935. How People Look at Pictures: A Study of the Psychology and Perception in Art. Chicago: University of Chicago Press.

  • Carpenter, Roger H. S. 2001. “Express Saccades: Is Bimodality a Result of the Order of Stimulus Presentation?Vision Research 41 (9): 11451151. https://doi.org/10.1016/S0042-6989(01)00007-4.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, Fang, Jianlong Zhou, Yang Wang, Kun Yu, Syed Z. Arshad, Ahmad Khawaji, and Dan Conway. 2016. Robust Multimodal Cognitive Load Measurement. Cham: Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chion, Michel. 1994. Audio-Vision: Sound on Screen. New York: Columbia University Press.

  • Crittenden, Roger. 1995. Film and Video Editing. 2nd ed. London: Routledge.

  • Crouzet, Sébastien M., Lyudmyla Y. Kovalenko, Simon Hviid del Pin, Morten Overgaard, and Niko A. Busch. 2017. “Early Visual Processing Allows for Selective Behavior, Shifts of Attention, and Conscious Visual Experience in Spite of Masking.” Consciousness and Cognition 54: 89100. https://doi.org//10.1016/j.concog.2017.01.021.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cuesta, Ubaldo, Luz Martínez-Martínez, and Jose I. Niño. 2018. “A Case Study in Neuromarketing: Analysis of the Influence of Music on Advertising Effectiveness through Eye-Tracking, Facial Emotion and GSR.” European Journal of Social Science Education and Research 5 (2): 7382. http://journals.euser.org/index.php/ejser/issue/view/198/ejser_v5_i2_18.pdf#page=73

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dalarna University Research Ethics Committee. 2008. Blankett för etisk egengranskning [Bill-of-Self-Audit]. Retrieved from http://www-old.du.se/sv/Om-Hogskolan/Organisation/Namnder/Forskningsetiska-namnden/Forskningsetik-/

  • Dancyger, Ken. 2007. The Technique of Film and Video Editing: History, Theory and Practice, 4th ed. London: Focal Press.

  • D'Ydewalle, Gery, Geert Desmet, and Johan Van Rensbergen. 1998. “Film Perception: The Processing of Film Cuts.” In Eye Guidance in Reading and Scene Perception, ed. Geoffrey Underwood, 357367. Oxford: Elsevier.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dykhoff, Klas. 2002. Ljudbild eller Synvilla? En Bok om Filmljud och Ljuddesign [Soundstage or illusion? A book on film sound and sound design]. Stockholm: Liber.

    • Search Google Scholar
    • Export Citation
  • Eg, Ragnhild, and Dawn M. Behne. 2015. “Perceived Synchrony for Realistic and Dynamic Audiovisual Events.” Frontiers in Psychology 6: 736. https://doi.org/10.3389/fpsyg.2015.00736.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Eisenstein, Sergei, and Dmitri Vasilyev. 1938. Alexander Nevsky. Moscow: Mosfilm.

  • Eisenstein, Sergei. 1947. The Film Sense. Translated and edited by Jay Leyda. San Diego: Harcourt & Brace.

  • Eriksson, Per Erik, Thorbjörn Swenberg, Xiayun Zhao, and Yvonne Eriksson. 2018. “How Gaze Time on Screen Impacts the Efficacy of Visual Instructions.” Heliyon 4 (6): e00660. https://doi.org/10.1016/j.heliyon.2018.e00660.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Escoffier, Nicolas, Darren Y. J. Sheng, and Annett Schirmer. 2010. “Unattended Musical Beats Enhance Visual Processing.” Acta Psychologica 135 (1): 1216. https://doi.org/10.1016/j.actpsy.2010.04.005.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fiebelkorn, Ian C., Mark A. Pinsk, and Sabine Kastner. 2018. “A Dynamic Interplay within the Frontoparietal Network Underlies Rhythmic Spatial Attention.” Neuron 99 (4): 842853. https://doi.org/10.1016/j.neuron.2018.07.038

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fogarty, Christine, and John A. Stern. 1989. “Eye Movements and Blinks: Their Relationship to Higher Cognitive Processes.” International Journal of Psychophysiology 8 (1): 3542. https://doi.org/10.1016/0167-8760(89)90017-2

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Foroughi, Cyrus K., Joseph T. Coyne, Ciara Sibley, Tatana Olson, Cory Moclaire, and Noelle Brown. 2017. “Pupil Dilation and Task Adaptation.” In International Conference on Augmented Cognition, 304311. Cham: Springer. https://doi.org/10.1007/978-3-319-58628-1_24.

    • Search Google Scholar
    • Export Citation
  • Fries, Pascal. 2015. “Rhythms for Cognition: Communication through Coherence.” Neuron 88 (1): 220235. https://doi.org/10.1016/j.neuron.2015.09.034

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gabrielsson, Alf. 1993. “The Complexities of Rhythm.” In Psychology and Music: The Understanding of Melody and Rhythm, ed. Thomas J. Tighe and W. Jay Dowling, 93120. New York: Lawrence Erlbaum.

    • Search Google Scholar
    • Export Citation
  • Ghirardelli, Thomas G., and Angélique A. Scharine. 2009. “Auditory-Visual Interactions.” Helmet-Mounted Displays: Sensation, Perception and Cognition Issues, ed. Clarence Rash, Michael Russo, Tomasz Letowski, and Elmar Schmeisser, 599618. Ft. Rucker, AL: U.S. Army Aeromedical Research Laboratory.

    • Search Google Scholar
    • Export Citation
  • Giard, Marie H., and Franck Peronnet. 1999. “Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study.” Journal of Cognitive Neuroscience 11 (5): 473490. https://doi.org/10.1162/089892999563544

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goldstein, Robert B., Russell L. Woods, and Eli Peli. 2007. “Where People Look when Watching Movies: Do All Viewers Look at the Same Place?Computers in Biology and Medicine 37: 957964. https://doi.org/10.1016/j.compbiomed.2006.08.018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Heeman, Jessica, Stefan Van der Stigchel, Douglas P. Munoz, and Jan Theeuwes. 2019. “Discriminating between Anticipatory and Visually Triggered Saccades: Measuring Minimal Visual Saccadic Response Time Using Luminance.” Journal of Neurophysiology 121 (6): 21012111. https://doi.org/10.1152/jn.00378.2018

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Henderson, John M. 2007. “Regarding Scenes.” Current Directions in Psychological Science 16 (4): 219222. https://doi.org/10.1111/j.1467-8721.2007.00507.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hennig, Holger, Ragnar Fleischmann, Anneke Fredebohm, York Hagmayer, Jan Nagler, Annette Witt, Fabian J. Theis, and Theo Geisel. 2011. “The Nature and Perception of Fluctuations in Human Musical Rhythms.” PloS one 6 (10): e26457.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hogendoorn, Hinze. 2016. “Voluntary Saccadic Eye Movements Ride the Attentional Rhythm.” Journal of Cognitive Neuroscience 28 (10): 16251635. https://doi.org/10.1162/jocn_a_00986

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Holcombe, Alex O. 2009. “Seeing Slow and Seeing Fast: Two Limits on Perception.” Trends in Cognitive Sciences 13 (5): 216221. https://doi.org/10.1016/j.tics.2009.02.005

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Holman, Tomlinson. 2010. Sound for Film and Television. New York: Focal Press.

  • Holmqvist, Kenneth, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van de Weijer. 2011. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.

    • Search Google Scholar
    • Export Citation
  • Hvelplund, Kristian T. 2011. “Allocation of Cognitive Resources in Translation: An Eye-Tracking and Key-Logging Study.” PhD thesis, Copenhagen Business School. https://openarchive.cbs.dk/handle/10398/8314.

    • Search Google Scholar
    • Export Citation
  • Ildirar, Sermin, Daniel T. Levin, Stephan Schwan, and Tim J. Smith. 2018. “Audio Facilitates the Perception of Cinematic Continuity by First-time Viewers.” Perception 47 (3): 276295. https://doi.org/10.1177/0301006617745782

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Iversen, John R., Aniruddh D. Patel, Brenda Nicodemus, and Karen Emmorey. 2015. “Synchronization to Auditory and Visual Rhythms in Hearing and Deaf Individuals.” Cognition 134: 232244. https://doi.org/10.1016/j.cognition.2014.10.018.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Janda, Isabelle. 2011. “Cutting Rhythms in Contemporary Music Videos and Its Possibility of Automation.” Marshall Plan Scholarship Research Paper. University of Applied Science St. Poelten. San Diego State University, San Diego, CA. https://static1.squarespace.com/static/559921a3e4b02c1d7480f8f4/t/5859d3d0ff7c50b1564cb08c/1482281938802/Jandal.pdf.

    • Search Google Scholar
    • Export Citation
  • Jones, Mari R., and Marilyn Boltz. 1989. “Dynamic Attending and Responses to Time.” Psychological Review 96 (3): 459491. https://doi.org/10.1037/0033-295X.96.3.459

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jumisko-Pyykkö, Satu, Jukka Häkkinen, and Göte Nyman. 2007. “Experienced Quality Factors: Qualitative Evaluation Approach to Audiovisual Quality.” In Multimedia on Mobile Devices 2007 6507: 65070M. International Society for Optics and Photonics. https://doi.org/10.1117/12.699797.

    • Search Google Scholar
    • Export Citation
  • Kinder, Annette, Martin Rolfs, and Reinhold Kliegl. 2008. “Sequence Learning at Optimal Stimulus–Response Mapping: Evidence from a Serial Reaction Time Task.” Quarterly Journal of Experimental Psychology 61 (2): 203209. https://doi.org/10.1080/17470210701557555.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koelewijn, Thomas, Hilde de Kluiver, Barbara G. Shinn-Cunningham, Adriana A. Zekveld, and Sophia E. Kramer. 2015. “The Pupil Response Reveals Increased Listening Effort when it is Difficult to Focus Attention.” Hearing Research 323: 8190. https://doi.org/10.1016/j.heares.2015.02.004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kramer, Arthur F. 1991. “Physiological Metrics of Mental Workload: A Review of Recent Progress.” In Multiple-Task Performance, ed. Diane L. Damos, 279328. London: Taylor & Francis. https://apps.dtic.mil/dtic/tr/fulltext/u2/a223701.pdf.

    • Search Google Scholar
    • Export Citation
  • Landau, Ayelet N. 2018. “Neuroscience: A Mechanism for Rhythmic Sampling in Vision.” Current Biology 28 (15): R830R832. https://doi.org/10.1016/j.cub.2018.05.081

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Landau, Ayelet N., and Pascal Fries. 2012. “Attention Samples Stimuli Rhythmically.” Current Biology 22 (11): 10001004. https://doi.org/10.1016/j.cub.2012.03.054

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lange, Elke B., Aleksandra Pieczykolan, Hans A. Trukenbrod, and Lynn Huestegge. 2018. “The Rhythm of Cognition: Effects of an Auditory Beat on Oculomotor Control in Reading and Sequential Scanning.” Journal of Eye Movement Research 11 (2). https://doi.org/10.16910/jemr.11.2.9.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Large, Edward W. 2000. “On Synchronizing Movements to Music.” Human Movement Science 19 (4): 527566. https://doi.org/10.1016/S0167-9457(00)00026-9

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lipscomb, Scott D., and Roger A. Kendall. 1994. “Perceptual Judgement of the Relationship between Musical and Visual Components in Film.” Psychomusicology: A Journal of Research in Music Cognition 13 (1–2): 6098. https://doi.org/10.1037/h0094101.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Luck, Geoff, and John A. Sloboda. 2009. “Spatio-Temporal Cues for Visually Mediated Synchronization.” Music Perception 26 (5): 465473. https://doi.org/10.1525/mp.2009.26.5.465

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marchant, Paul, David Raybould, Tony Renshaw, and Richard Stevens. 2009. “Are You Seeing What I'm Seeing? An Eye-Tracking Evaluation of Dynamic Scenes.” Digital Creativity 20: 153163. https://doi.org/10.1080/14626260903083611.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • May, Jon, Michael P. Dean, and Philip J. Barnard. 2003. “Using Film Cutting Techniques in Interface Design.” Human-Computer Interactions 18 (4): 325372. https://doi.org/10.1207/S15327051HCI1804_1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moens, Bart, Chris Muller, Leon van Noorden, Marek Franeˇk, Bert Celie, Jan Boone, Jan Bourgois, and Marc Leman. 2014. “Encouraging Spontaneous Synchronisation with D-Jogger, an Adaptive Music Player that Aligns Movement and Music.” PloS one 9 (12): e114234. https://doi.org/10.1371/journal.pone.0114234.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Murch, Walter. 2001. In the Blink of the Eye: A Perspective on Film Editing. 2nd ed. Los Angeles: Silman-James Press.

  • Orchard-Mills, Emily, Erik Van der Burg, and David Alais. 2013. Amplitude-modulated Auditory Stimuli Influence Selection of Visual Spatial Frequencies. Journal of Vision 13 (3): 6. https://doi.org/10.1167/13.3.6

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Osberger, Wilfried, and Ann M. Rohaly. 2001. “Automatic Detection of Regions of Interest in Complex Video Sequences.” In Proceedings of SPIE: Human Vision and Electronic Imaging VI 4299: 361372. https://doi.org/10.1117/12.429506.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Palmer, Stephen E. 1999. Vision Science: Photons to Phenomenology. Boston, MA: MIT press.

  • Pearlman, Karen. 2009. Cutting Rhythms: Shaping the Film Edit. Burlington, MA: Focal Press.

  • Pearlman, Karen. 2017. “Editing and Cognition: Beyond Continuity.” Projections 11 (2): 6786. https://doi.org/10.3167/proj.2017.110205

  • Posner, Michael I., Mary J. Nissen, and William C. Ogden. 1978. “Attended and Unattended Processing Modes: The Role of Set for Spatial Location.” In Modes of Perceiving and Processing Information, ed. Herbert L. Pick Jr. and Elliot Saltzman, 137158. Hillsdale, NJ: Erlbaum.

    • Search Google Scholar
    • Export Citation
  • Posner, Michael I. 1980. Orienting of Attention. Quarterly Journal of Experimental Psychology 32 (1): 325.

  • Poulaki, Maria. 2015. “Brain Science and Film Theory: Reassessing the Place of Cognitive Discontinuity in Cinema.” Projections 9 (1): 2342. https://doi.org/10.3167/proj.2015.090103

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Repp, Bruno H., and Yi-Huang Su. 2013. “Sensorimotor Synchronization: A Review of Recent Research (2006–2012).” Psychonomic Bulletin & Review 20 (3): 403452. https://doi.org/10.3758/s13423-012-0371-2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rose, Susan A., Judith F. Feldman, Jeffery J. Jankowski, and Donna M. Caro. 2002. “A Longitudinal Study of Visual Expectation and Reaction Time in the First Year of Life.” Child Development 73 (1): 4761. www.jstor.org/stable/3696430

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Seeber, Kilian G. 2013. “Cognitive Load in Simultaneous Interpreting: Measures and Methods.” Target 25 (1): 1832. https://doi.org/10.1075/target.25.1.03see

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shafiq, R., Geoffrey W. Stuart, Jennifer Sandbach, Paul Maruff, and Jon Currie, 1998. “The Gap Effect and Express Saccades in the Auditory Modality.” Experimental Brain Research 118 (2): 221229. https://doi.org/10.1007/s002210050275

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shimamura, Arthur P., ed. 2013. Psychocinematics: Exploring Cognition at the Movies. Oxford: Oxford University Press.

  • Shimamura, Arthur P., Brendan I. Cohn-Sheehy, Brianna L. Pogue, and Thomas A. Shimamura. 2015. “How Attention Is Driven by Film Edits: A Multimodal Experience.” Psychology of Aesthetics, Creativity, and the Arts 9 (4): 417422. https://doi.org/10.1037/aca0000025

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, Tim J. 2005. “An Attentional Theory of Continuity Editing.” PhD thesis, University of Edinburgh. https://era.ed.ac.uk/bitstream/handle/1842/1076/smith_ATOCE_0506.pdf.

    • Search Google Scholar
    • Export Citation
  • Smith, Tim J. 2012. “The Attentional Theory of Cinematic Continuity.” Projections 6 (1): 127. https://doi.org/10.3167/proj.2012.060102

  • Smith, Tim J. 2014. “Audiovisual Correspondences in Sergei Eisenstein's Alexander Nevsky: A Case Study in Viewer Attention.” In Cognitive Media Theory, ed. Paul Taberham and Ted Nannicelli. Abingdon: Taylor & Francis.

    • Search Google Scholar
    • Export Citation
  • Smith, Tim J., and John M. Henderson. 2008. “Edit Blindness: The Relationship between Attention and Global Change Blindness in Dynamic Scenes.” Journal of Eye Movement Research 2 (2): 117. https://doi.org/10.16910/jemr.2.2.6

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, Tim J., and John M. Henderson. 2009. “Facilitation of Return during Scene Viewing.” Visual Cognition 17 (6/7): 10831108. https://doi.org/10.1080/13506280802678557.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, Tim J., Daniel Levin, and James E. Cutting. 2012. “A Window on Reality: Perceiving Edited Moving Images.” Current Directions in Psychological Science 21 (2): 107113. https://doi.org/10.1177/0963721412437407.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Smith, Tim J., Martyn Whitwell, and John Lee. 2006. “Eye Movements and Pupil Dilation during Event Perception.” In Proceedings of the 2006 Symposium on Eye Tracking Research & Applications, 48. https://dl.acm.org/doi/pdf/10.1145/1117309.1117333.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Snowden, Robert J., Peter Thompson, and Tom Troscianko. 2012. Basic Vision: An Introduction to Visual Perception. Oxford: Oxford University Press.

    • Search Google Scholar
    • Export Citation
  • Stefan, Julia, and Andrew R. Brown. 2015. “Generative Music Video Composition: Using Automation to Extend Creative Practice.” Digital Creativity 26 (2): 110120. https://doi.org/10.1080/14626268.2014.932289.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stevenson, Ryan A., and Mark T. Wallace. 2013. “Multisensory Temporal Integration: Task and Stimulus Dependencies.” Experimental Brain Research 227 (2): 249261. https://doi.org/10.1007/s00221-013-3507-3.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Su, Yi-Huang. 2014. “Peak Velocity as a Cue in Audiovisual Synchrony Perception of Rhythmic Stimuli.” Cognition 131 (3): 330344. https://doi.org/10.1016/j.cognition.2014.02.004

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Swenberg, Thorbjörn. 2017a. “A Film Editor's Visual Intentions and Use of Perceptual Phenomena in Designing Film Edits: An Observational Study.” International Journal of Visual Design 11 (3): 117. https://doi.org/10.18848/2325-1581/CGP/v11i03/1-17

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Swenberg, Thorbjörn. 2017b. “Framing the Gaze: (Audio-) Visual Design Intentions and Perceptual Considerations in Film Editing.” PhD thesis, Mälardalen University, Eskilstuna. https://www.diva-portal.org/smash/record.jsf?pid=diva2 percent3A1110031&dswid = -2659.

    • Search Google Scholar
    • Export Citation
  • Swenberg, Thorbjörn, and Per Erik Eriksson. 2018. “Effects of Continuity or Discontinuity in Actual Film Editing.” Empirical Studies of the Arts 36 (2): 222246. https://doi.org/10.1177/0276237417744590.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Torre, Kjerstin, Manuel Varlet, and Vivien Marmelat. 2013. “Predicting the Biological Variability of Environmental Rhythms: Weak or Strong Anticipation for Sensorimotor Synchronization?Brain and Cognition 83 (3): 342350. https://doi.org/10.1016/j.bandc.2013.10.002

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vatakis, Argiro, and Charles Spence. 2006. “Audiovisual Synchrony Perception for Music, Speech, and Object Actions.” Brain Research 1111 (1): 134142. https://doi.org/10.1016/j.brainres.2006.05.078.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vernallis, Carol. 2001. “The Kindest Cut: Functions and Meanings of Music Video Editing.” Screen 42 (1): 2148. https://doi.org/10.1093/screen/42.1.21

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Vickers, Stephen, Howell Istance, and Matthew Smalley. 2010. “EyeGuitar: Making Rhythm-Based Music Video Games Accessible Using Only Eye Movements.” In Proceedings of the 7th International Conference on Advances in Computer Entertainment Technology, 3639. ACM. https://dl.acm.org/doi/pdf/10.1145/1971630.1971641.

    • Search Google Scholar
    • Export Citation
  • Vroomen, Jean, and Beatrice De Gelder. 2000. “Sound Enhances Visual Perception: Cross- Modal Effects of Auditory Organization on Vision.” Journal of Experimental Psychology: Human Perception and Performance 26 (5): 15831590. https://doi.org/10.1037/0096-1523.26.5.1583.

    • Search Google Scholar
    • Export Citation
  • White, Peter A. 2018. “Is Conscious Perception a Series of Discrete Temporal Frames?” Consciousness and Cognition 60: 98126. https://doi.org/10.1016/j.concog.2018.02.012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wingstedt, Johnny. 2008. “Beat Sync for Video.” [Unpublished lecture material.] Stockholm: Kungliga Musikhögskolan [Royal College of Music].

    • Search Google Scholar
    • Export Citation
  • Wooley, Brooke E., and David S. March. 2014. “Exploring the Influence of Audio in Directing Visual Attention during Dynamic Content.” In Proceedings of the Symposium on Eye Tracking Research and Applications, 187190. ACM. https://dl.acm.org/doi/pdf/10.1145/2578153.2578180.

    • Search Google Scholar
    • Export Citation
  • Wyatt Hilary, and Tim Amyes. 2005. Audio Post Production for Television and Film: An Introduction to Technology and Techniques. Burlington, MA: Focal Press.

    • Search Google Scholar
    • Export Citation
  • Wöllner, Clemens, Frederik J. A. Deconinck, Jim Parkinson, Michael J. Hove, and Peter E. Keller. 2012. “The Perception of Prototypical Motion: Synchronization is Enhanced with Quantitatively Morphed Gestures of Musical Conductors.” Journal of Experimental Psychology: Human Perception and Performance 38: 13901403. https://doi.org/10.1037/a0028130.

    • Search Google Scholar
    • Export Citation
  • Yarbus, Alfred L. 1967. Eye Movements and Vision. New York: Plenum Press.

  • Zacks, Jeffrey M. 2015. Flicker: Your Brain on Movies. Oxford: Oxford University Press.

  • Zu, Tianlong, John Hutson, Lester C. Loschky, and N. Sanjay Rebello. 2018. “Use of Eye-Tracking Technology to Investigate Cognitive Load Theory.” In 2017 PERC Proceedings, 472475. https://doi.org/10.1119/perc.2017.pr.113.

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 3265 1757 299
Full Text Views 135 5 0
PDF Downloads 182 11 0