We discuss the structure of rapidity divergences that are presented in the soft factors of transverse momentum dependent (TMD) factorization theorems. To provide the discussion on the most general level we consider soft factors for multi-parton scattering. We show that the rapidity divergences are result of the gluon exchanges with the distant transverse plane, and are structurally equivalent to the ultraviolet divergences. It allows to formulate and to prove the renormalization theorem for rapidity divergences. The proof is made with the help the conformal transformation which maps rapidity divergences to ultraviolet divergences. The theorem is the systematic form of the factorization of rapidity divergences, which is required for the definition of TMD parton distributions. In particular, the definition of multi parton distributions is presented. The equivalence of ultraviolet and rapidity divergences leads to the exact relation between soft and rapidity anomalous dimensions. Using this relation we derive the rapidity anomalous dimension at the three-loop order.