We determine the axial, scalar and tensor isovector charges of the nucleon, sigma and cascade baryons as well as the difference between the up and down quark masses, mu−md. We employ gauge ensembles with Nf=2+1 nonperturbatively improved Wilson fermions at six values of the lattice spacing in the range a≈(0.039–0.098)  fm, generated by the coordinated lattice simulations (CLS) effort. The pion mass Mπ ranges from around 430 MeV down to a near physical value of 130 MeV and the linear spatial lattice extent L varies from 6.5M−1π to 3.0M−1π, where LMπ≥4 for the majority of the ensembles. This allows us to perform a controlled interpolation/extrapolation of the charges to the physical mass point in the infinite volume and continuum limit. Investigating SU(3) flavor symmetry, we find moderate symmetry breaking effects for the axial charges at the physical quark mass point, while no significant effects are found for the other charges within current uncertainties.