This study aimed to validate the short-story-task (SST) based on Dodell-Feder et al. as an instrument to quantify the ability of mentalizing and to differentiate between non-autistic adults and autistic adults, who may have acquired rules to interpret the actions of non-autistic individuals. Autistic (N = 32) and non-autistic (N = 32) adult participants were asked to read "The End of Something " by Ernest Hemingway and to answer implicit and explicit mentalizing questions, and comprehension questions. Furthermore, verbal and nonverbal IQ was measured and participants were asked how much fiction they read each month. Mentalizing performance was normally distributed for autistic and non-autistic participants with autistic participants scoring in the lower third of the distribution. ROC (receiver operator curve) analysis revealed the task to be an excellent discriminator between autistic and non-autistic participants. A linear regression analysis identified number of books read, years of education and group as significant predictors. Overall, the SST is a promising measure of mentalizing. On the one hand, it differentiates among non-autistic individuals and on the other hand it is sensitive towards performance differences in mentalizing among autistic adults. Implications for interventions are discussed.