In scanning tunneling experiments on semiconductor surfaces, the energy scale within the tunneling junction is usually unknown due to tip-induced band bending. Here, we experimentally recover the zero point of the energy scale by combining scanning tunneling microscopy with Kelvin probe force spectroscopy. With this technique, we revisit shallow acceptors buried in GaAs. Enhanced acceptor-related conductance is observed in negative, zero, and positive band-bending regimes. An Anderson-Hubbard model is used to rationalize our findings, capturing the crossover between the acceptor state being part of an impurity band for zero band bending and the acceptor state being split off and localized for strong negative or positive band bending, respectively.