We identify the dominant source for low-frequency spin qubit splitting noise in a highly isotopically-purified silicon device with an embedded nanomagnet and a spin echo decay time Techo2 = 128 µs. The power spectral density (PSD) of the charge noise explains both, the clear transition from a 1/f2- to a 1/f-dependence of the splitting noise PSD as well as the experimental observation of a decreasing time-ensemble spin dephasing time, from T∗2≈ 20 µs, with increasing measurement time over several hours. Despite their strong hyperfine contact interaction, the few 73Ge nuclei overlapping with the quantum dot in the barrier do not limit T∗2, likely because their dynamics is frozen on a few hours measurement scale. We conclude that charge noise and the design of the gradient magnetic field are the key to further improve the qubit fidelity in isotopically purified 28Si/SiGe.