We show electric field control of the spin accumulation at the interface of the oxide semiconductor Nb-SrTiO3 with Co/AlOx spin injection contacts at room temperature. The in-plane spin lifetime tau(parallel to), as well as the ratio of the out-of-plane to in-plane spin lifetime tau(perpendicular to)/tau(parallel to), is manipulated by the built-in electric field at the semiconductor surface, without any additional gate contact. The origin of this manipulation is attributed to Rashba spin orbit fields (SOFs) at the Nb-SrTiO3 surface and shown to be consistent with theoretical model calculations based on SOF spin flip scattering. Additionally, the junction can be set in a high or low resistance state, leading to a nonvolatile control of tau(perpendicular to)/tau(parallel to), consistent with the manipulation of the Rashba SOF strength. Such room temperature electric field control over the spin state is essential for developing energy-efficient spintronic devices and shows promise for complex oxide based (spin) electronics.