In chemistry and physics symmetry principles are all important, for example, leading to the selection rules governing optical transitions. We have investigated the influence of the molecular symmetry on the surface potential landscape of molecules in the limit of weak molecule-substrate binding. For this purpose, the induced lateral motion of Cu(II)-tetraazaphthalocyanine molecules, for which four symmetry distinct isomers exist, on NaCl(100) was studied by scanning tunneling microscopy. This nonthermal diffusion induced by inelastic excitations is found to be qualitatively different for all four symmetry distinct isomers, demonstrating that symmetry governs the surface potential landscape.