Understanding the decoherence of electron spins in semiconductors due to their interaction with nuclear spins is of fundamental interest as they realize the central spin model and of practical importance for using them as qubits. Interesting effects arise from the quadrupolar interaction of nuclear spins with electric field gradients, which have been shown to suppress diffusive nuclear spin dynamics and might thus enhance electron spin coherence. Here we show experimentally that for gate-defined GaAs quantum dots, quadrupolar broadening of the nuclear Larmor precession reduces electron spin coherence by causing faster decorrelation of transverse nuclear fields. However, this effect disappears for appropriate field directions. Furthermore, we observe an additional modulation of coherence attributed to an anisotropic electronic g-tensor. These results complete our understanding of dephasing in gated quantum dots and point to mitigation strategies. They may also help to unravel unexplained behaviour in self-assembled quantum dots and III-V nanowires.