Background: Habitat fragmentation is considered to be a main reason for decreasing genetic diversity of plant species. However, the results of many fragmentation studies are inconsistent. This may be due to the influence of habitat conditions, having an indirect effect on genetic variation via reproduction. Consequently we took a comparative approach to analyse the impact of habitat fragmentation and habitat conditions on the genetic diversity of calcareous grassland species in this study. We selected five typical grassland species (Primula veris, Dianthus carthusianorum, Medicago falcata, Polygala comosa and Salvia pratensis) occurring in 18 fragments of calcareous grasslands in south eastern Germany. We sampled 1286 individuals in 87 populations and analysed genetic diversity using amplified fragment length polymorphisms. Additionally, we collected data concerning habitat fragmentation (historical and present landscape structure) and habitat conditions (vegetation structure, soil conditions) of the selected study sites. The whole data set was analysed using Bayesian multiple regressions. Results: Our investigation indicated a habitat loss of nearly 80% and increasing isolation between grasslands since 1830. Bayesian analysis revealed a significant impact of the historical landscape structure, whereas habitat conditions played no important role for the present-day genetic variation of the studied plant species. Conclusions: Our study indicates that the historical landscape structure may be more important for genetic diversity than present habitat conditions. Populations persisting in abandoned grassland fragments may contribute significantly to the species' variability even under deteriorating habitat conditions. Therefore, these populations should be included in approaches to preserve the genetic variation of calcareous grassland species.