1. Contrastive Flow Matching

In conditional flow, the model may collapse multiple diverse outputs into a single trajectory, yielding samples that lack the expected specificity and diversity for each condition.

Contrastive Flow Matching (CFM) applies a pairwise loss term between samples in a training batch: for each positive sample from the batch, we randomly sample a negative counterpart.

$${ℒ}^{\mathrm{\Delta }\text{FM}}\left(𝜃\right)=𝐸\left[{\Vert {𝑣}_{𝜃}\left({𝑥}_{𝑡},𝑡,𝑦\right)-\left({\dot{𝛼}}_{𝑡}\widehat{𝑥}+{\dot{𝜎}}_{𝑡}𝜀\right)\Vert }^2-𝜆{\Vert {𝑣}_{𝜃}\left({𝑥}_{𝑡},𝑡,𝑦\right)-\left({\dot{𝛼}}_{𝑡}\widetilde{𝑥}+{\dot{𝜎}}_{𝑡}\widetilde{𝜀}\right)\Vert }^2\right]$$

2. Dispersive Loss

Intuitively, Dispersive Loss encourages internal representations to spread out in the hidden space, analogous to the repulsive effect in contrastive learning. Meanwhile, the original regression loss (e.g., denoising) naturally serves as an alignment mechanism, eliminating the need to manually define positive pairs as in contrastive learning.

$$𝒳={𝐸}_{{𝑥}_{𝑖}\in 𝑋}\left[{ℒ}_{\text{Diff }}\left({𝑥}_{𝑖}\right)\right]+𝜆{ℒ}_{\text{ Disp }}\left(𝑋\right)$$

This method is self-contained and minimalist. Conceptually, Dispersive Loss can be derived from any existing contrastive loss by appropriately removing the positive terms. In this regard, the term “Dispersive Loss” does not refer to a specific implementation, but rather to a general class of objectives that encourage dispersion. We introduce several variants of Dispersive Loss functions in the following.

Variant	Contrastive	Dispersive
InfoNCE	$𝐷\left({𝑧}_{𝑖},{𝑧}_{𝑖}^{+}\right)/𝜏+\log {\sum }_{𝑗}\exp \left(-𝐷\left({𝑧}_{𝑖},{𝑧}_{𝑗}\right)/𝜏\right)$	$\log {𝐸}_{𝑖𝑗}\left[\exp \left(-𝐷\left({𝑧}_{𝑖},{𝑧}_{𝑗}\right)/𝜏\right)\right)]$
Hinge	${𝐷\left({𝑧}_{𝑖},{𝑧}_{𝑖}^{+}\right)}^2+{𝐸}_{𝑗}\left[{\max \left(0,𝜀-𝐷\left({𝑧}_{𝑖},{𝑧}_{𝑗}\right)\right)}^2\right]$	${𝐸}_{𝑖𝑗}\left[{\max \left(0,𝜀-𝐷\left({𝑧}_{𝑖},{𝑧}_{𝑗}\right)\right)}^2\right]$
Covariance	${\left(1-{\text{ Cov}}_{𝑖𝑖}\right)}^2+𝑤{\sum }_{𝑖\ne 𝑗}{\text{ Cov}}_{𝑖𝑗}^2$	${\sum }_{𝑖𝑗}{\text{ Cov}}_{𝑖𝑗}^2$

Dispersive vs. Contrastive. We compare different variants of Dispersive Loss with their contrastive counterparts. To apply a contrastive loss, two views are sampled for each training example to form a positive pair. We study two strategies for adding noise to both views: (i) sampling noise independently for each view, following the generative model’s noising policy; or (ii) sampling noise for the first view according to the same policy, and then restricting the second view’s noise level to differ by at most 0.005. Moreover, to avoid doubling training epochs due to two-view sampling, we only apply the denoising loss to the first view, for fair comparisons with both the baseline and our method.

Table shows that when using independent noise, the contrastive loss fails to improve generation quality in all cases studied. We hypothesize that aligning two views with substantially different noise levels impairs learning. As evidence, Table shows that contrastive loss with restricted noise leads to improvements over the baseline in three of the four evaluated variants. These experiments suggest that contrastive learning is sensitive to the choice of data augmentation. Noising, which serves as a built-in form of augmentation in diffusion models, further complicates the problem. While contrastive learning can be mildly beneficial, the design of the additional view and the coupling of two views may limit its application.

References

1. Contrastive Flow Matching
2. Diffuse and Disperse: Image Generation with Representation Regularization