In the past, current-distance spectroscopy has been widely applied to determine variations of the work function at surfaces. While for homogeneous sample areas this technique is commonly accepted to yield at least qualitative results, its applicability to atomic-scale variations has not been proven neither right nor wrong. Here we benchmark measurements of the current-distance decay constant against the well established Kelvin probe force spectroscopy for four distinctly different cases with atomic-scale variations of the local contact potential. The two techniques yield quite different results. Whereas the maps of the current-distance decay constant are consistent with being topographical artifacts, the Kelvin probe force spectroscopy maps show variations of the local contact potential difference in agreement with expected surface dipoles. This comparison clarifies that maps of the current-distance decay constant are not suited to directly characterize contact potential variations at surfaces on atomic length scales.