We present a perturbative result for the temporal evolution of the fidelity of the quantum kicked rotor, i.e. the overlap of the same initial state evolved with two slightly different kicking strengths, for kicking periods close to a principal quantum resonance. Based on a pendulum approximation, we describe the fidelity for rotational orbits in the pseudo-classical phase space of a corresponding classical map. Our results are compared to numerical simulations indicating the range of applicability of our analytical approximation.