According to the quantum chaos paradigm, the nature of a system's classical dynamics, whether integrable or chaotic, is universally reflected in the fluctuations of its quantum spectrum. However, since many-body spectra in the mean field limit are composed of independent single-particle energy levels, their spectral fluctuations always display Poissonian behavior and hence cannot be used to distinguish underlying chaotic from integrable single-particle dynamics. We demonstrate that this distinction can, instead, be revealed from the mean many-body level density (at fixed energy) and its variance after averaging over ensembles representing different types of single-particle dynamics. This is in strong contrast to the energy-averaged mean level density (of a given system) that is assumed not to carry such information and is routinely removed to focus on universal signatures. To support our claim we systematically analyze the role of single-particle level correlations, that enter through Poisson and random matrix statistics (of various symmetry classes) into the ensemble-averaged density of states and its variance, contrasting bosonic and fermionic many-body systems. Our analytical study, together with extensive numerical simulations for systems with N≥5 particles consistently reveal significant differences (up to an order of magnitude for fermions and even larger for bosons) in the mean many-body level densities, depending on the nature of the underlying dynamics. Notably, in the fermionic case Poisson-type single-particle level fluctuations precisely cancel contributions from indistinguishability, such that the average many-body spectral density equals the (Thomas-Fermi) volume term. We further highlight the difference between the mean level density and its variance as functions of the total energy E and the excitation energy Q.