We report the observation of the anisotropic polar magneto-optical Kerr effect in thin layers of epitaxial Fe/GaAs(001) at room temperature. A clear twofold symmetry of the Kerr rotation angle depending on the orientation of the linear polarization of the probing laser beam with respect to the crystallographic directions of the sample is detected for ultrathin magnetic films saturated out of the film plane. The amplitude of the anisotropy decreases with increasing Fe film thickness, suggesting that the interfacial region is the origin of the anisotropy. The twofold symmetry is fully reproduced by model calculations based on an interference of interfacial Bychkov-Rashba and Dresselhaus spin-orbit coupling.