We study disorder-induced spectral correlations and their effect on the magnetic susceptibility of mesoscopic quantum systems in the non-diffusive regime. By combining a diagrammatic perturbative approach with semiclassical techniques we perform impurity averaging for non-translational invariant systems. This allows us to study the crossover from clean to diffusive systems. As an application we consider the susceptibility of non-interacting electrons in a ballistic microstructure in the presence of weak disorder. We present numerical results for a square billiard and approximate analytic results for generic chaotic geometries. We show that for the elastic mean free path { l } larger than the system size, there are two distinct regimes of behaviour depending on the relative magnitudes of { l} and an inelastic scattering length.