The perturbation theory for critical points of causal variational principles is developed. We first analyze the class of perturbations obtained by multiplying the universal measure by a weight function and taking the push-forward under a dif-
feomorphism. Then the constructions are extended to convex combinations of such measures, leading to perturbation expansions for the mean and the fluctuation of the measure, both being coupled in higher order perturbation theory. It is explained how our methods and results apply to the causal action principle for causal fermion systems. It is shown how the perturbation expansion in the continuum limit and the
effect of microscopic mixing are recovered in specific limiting cases.