Dissociating external features from internal structures in visual segmentation of actions

The human mind tends to represent continuous experience as discrete events, imposing “event boundaries” on incoming streams of sensory data. This phenomenon, known as event segmentation, is not just a function of top-down decisions about where events begin and end – event boundaries appear to structure attention and perception as well. How are event boundaries represented perceptually? Previous studies on event structure typically employ remarkable changes of physical features at event boundaries, e.g., walking through a doorway, large shifts in objects, figures and scenes, salient motion cues, and disruptions of visual statistics. This raises an important question: Is the perceptual representation of an event boundary exclusively driven by processing salient, lower-level physical changes and motion dynamics? Or might higher-level semantic structure also shape how we perceive event boundaries? Here we attempt to disentangle low-level spatiotemporal features of continuous visual input versus high-level representations of natural action structure. Across six pre-registered experiments, we asked participants to detect subtle disruptions in 20 short, individual actions (e.g., kicking a ball, stepping over an obstacle, throwing a frisbee, etc., generated as motion-capture-based simple animations, static images, and point-light biological motion displays), which were presented either in a recognizable intact form or in a distorted manner that only preserved low-level spatiotemporal dynamics and visual features. Results consistently demonstrated that visual detection of subtle disruptions was weaker at action boundaries (i.e., the transition between discrete steps within the action) relative to non-boundaries, extending previous findings to naturalistic action perception. Crucially, these perceptual effects were driven by both lower-level visual features and by high-level information of action structure. Thus, automatic and rapid perceptual segmentation of actions is likely structured in time by both external cues inherent to the stimulus and our internal models of the world.