Central spin models describe several types of solid state nanostructures which are presently considered as possible building blocks of future quantum information processing hardware. From a theoretical point of view, a key issue remains the treatment of the flip-flop terms in the Hamiltonian in the presence of a magnetic field. We systematically study the influence of these terms, both as a function of the field strength and the size of the spin baths. We find crucial differences between initial states with central spin configurations of high and such of low polarizations. This has strong implications with respect to the influence of a magnetic field on the flip-flop terms in central spin models of a single and more than one central spin. Furthermore, the dependencies on bath size and field differ from those anticipated so far. Our results might open the route for the systematic search for more efficient perturbative treatments of central spin problems.