The lateral occipitotemporal cortex (LOTC) has been suggested to host action representations that are thought to
contribute to accessing meaning of observed actions. In line with this proposal, previous studies have shown spatially
overlapping representations of various action-related
features including objects, scenes, social properties, and kinematics,
as well as abstract representations pertaining to action semantics. Less is known, however, about the way in
which these features are integrated during action recognition. To address this question, we aimed to examine the
temporal order in which different action-related
features emerge in the EEG signal using EEG-based
representational
similarity analysis (RSA). Additionally, to investigate the spatiotemporal characteristics of these neural representations,
we employed fMRI-EEG
fusion. Static images spanning 27 everyday actions (e.g., riding a bike, washing dishes,
brushing hair) were shown to participants (N = 24) in a delayed matching task, in a total of 648 trials (using different
combinations of locations, actors, and viewpoints). Participants were asked to judge stimuli with respect to the type
of action, location, and actor identity. Temporally specific neural representational dissimilarity matrices (RDMs) pertaining
to neural representations of action categories were generated from 64-channel
EEG recordings using time-resolved
RSA. These were correlated with behavioral RDMs pertaining to semantic action (dis)similarities, revealing a
time course of neural semantic action representations. Additionally, we considered RDMs capturing (dis)similarities
between actions in terms of involved body postures, scenes, social aspects, and lower-level
visual information.
Results suggest a temporally ordered hierarchical buildup of neural representations related to visual, contextual,
body-related,
and semantic action information. fMRI-EEG
fusion analysis further suggests that striate and extrastriate
areas along the lateral visual pathway encode lower-level
visual and body-related
properties of actions, and that contextual
and semantic information is integrated in the LOTC. These outcomes provide a spatiotemporal characterization
of the neural processes enabling humans to recognize goal-directed
actions.