Context Land use change reduced calcareous grasslands throughout Europe during the last decades. Subsequent fragmentation and habitat deterioration led, moreover, to a massive biodiversity decline. To counteract this alarming development, a clear understanding of genetic variation patterns, as fundamental level of biodiversity, becomes inevitable. Objectives The aim of our study was to identify the drivers of genetic variation in common calcareous grassland plant species. More specifically, we tested whether genetic diversity or differentiation ofAsperula cynanchica,Campanula rotundifolia, andLinum catharticumdepend on habitat age, landscape structure, habitat quality, and/or population size. Methods We investigated 912 individuals, 304 per study species, from 19 calcareous grasslands across the Swabian Alb in Baden-Wurttemberg (Germany) using AFLP analyses. Results We observed no significant influence of habitat age on genetic diversity and differentiation. Habitat quality also had no impact on genetic diversity and population size only showed weak effects. However, genetic diversity strongly depended on landscape structure represented by distance to the nearest settlement, total area of surrounding calcareous grasslands, and their connectivity. Conclusions Migratory sheep herding is considered as main land use in calcareous grasslands on the Swabian Alb and thus, landscape structures in the study region may describe movement patterns of grazing livestock. In this study, genetic variation in calcareous grassland populations was strongly affected by surrounding landscape structures and subsequent grazing patterns. Therefore, we assume that moderate grazing intensities over the long term may increase levels of genetic diversity, whereas periods of overgrazing or abandonment could lower genetic diversity.