Finding a reliable and objective measure of individual differences in mental imagery across sensory modalities is difficult, with measures relying on self-report scales or focusing on one modality alone. Based on the idea that mental imagery involves multimodal sensorimotor simulations, a mental comparison task (MCT) was developed across three studies and tested on adults (n = 96, 345, and 448). Analyses examined: (a) the internal consistency of the MCT, (b) whether lexical features of the MCT stimuli (word length and frequency) predicted performance, (c) whether the MCT related to two widely used self-report scales, (d) response latencies and accuracies across the visual, auditory, and tactile modalities, and (e) whether MCT performance was independent of processing speed. The MCT showed evidence of reliability and validity. Responses were fastest and most accurate for the visual modality, followed by the auditory and tactile. However, consistent with the idea that self-report questionnaires index a different aspect of mental imagery, the MCT showed minimal correlations with self-report imagery. Finally, relations between MCT scales remained strong after controlling for processing speed. Findings are discussed in relation to current understanding and measurement of mental imagery.