The recognition of actions presented as static photographs has been suggested to be modulated by the scene context—or image background —in which an action is presented, particularly for actions that are hard to recognize. This suggests scene context as an important feature when retrieving action representations, and poses the question whether scene context directly modulates the representational space of actions. Therefore, in this preregistered virtual reality (VR) study, we investigated in a group of N = 24 healthy participants how scene context by means of the (virtual) environment affects the representation of actions that do or do not correspond to the current environment. Participants performed multi-arrangement tasks on a set of kitchen and workshop actions while the virtual environment was manipulated to represent either a kitchen, workshop or neutral environment. Representational structure was estimated from the multi-arrangement tasks separately for each virtual environment and action domain (kitchen, workshop). Results showed increased consistency of the representation of kitchen actions over participants when the task was performed in a congruent (i.e., kitchen) compared with an incongruent (i.e., workshop) or neutral virtual environment, however, the same effect was not observed for workshop actions. Representational structure, however, was not affected by the environment. In accordance with previous studies on representational changes in domain-specific experts within various action and object domains the results suggest that an agent’s environment may modulate the action space for corresponding actions by increasing robustness of represented familiar actions. These findings contribute to our understanding of contextual effects on mental representations within the domain of actions and beyond.