We study double parton distributions with flavor interference in the nucleon and compare them with previous results for the flavor diagonal case. We investigate both unpolarized and polarized partons. We compare our lattice results with those obtained from the simple description of the proton in terms of an SU(6) symmetric three-quark wave function and find that this description fails for both flavor and polarization dependence. We also derive and test a factorization ansatz for the unpolarized flavor interference distribution in terms of single-parton distributions and find that this ansatz fails to a large extent.