We construct a unified semiclassical theory of charge and spin transport in chaotic ballistic and disordered diffusive mesoscopic systems with spin-orbit interaction. Neglecting dynamic effects of spin-orbit interaction, we reproduce the random matrix theory results that the spin conductance fluctuates universally around zero average. Incorporating these effects into the theory, we show that geometric correlations generate finite average spin conductances, but that they do not affect the charge conductance to leading order. The theory, which is confirmed by numerical transport calculations, allows us to investigate the entire range from the weak to the previously unexplored strong spin-orbit regime, where the spin rotation time is shorter than the momentum relaxation time.