1. MeanFlow: One-step Generative Modeling via Average Velocity

MeanFlow introduces a principled framework for one-step generative modeling by fundamentally reconsidering what velocity field to model. Instead of learning instantaneous velocities like traditional Flow Matching, MeanFlow directly models average velocities - the displacement over a time interval divided by that interval. This seemingly simple change leads to a well-defined mathematical identity that enables stable, curriculum-free training and achieves state-of-the-art one-step generation results.

1.1. The Core Insight: Average Velocity

1.1.1. Intuition

The key insight of MeanFlow is that instead of modeling the instantaneous velocity $𝑣\left({𝑧}_{𝑡},𝑡\right)$ that describes the tangent direction at each point, we can directly model the average velocity $𝑢\left({𝑧}_{𝑡},𝑟,𝑡\right)$ that represents the total displacement from time $𝑟$ to time $𝑡$ divided by the time interval.

1.1.2. Mathematical Definition

The average velocity is defined as:

$$𝑢\left({𝑧}_{𝑡},𝑟,𝑡\right)=\frac{1}{𝑡-𝑟}{\int }_{𝑟}^{𝑡}𝑣\left({𝑧}_{𝜏},𝜏\right)𝑑𝜏$$

This represents the displacement between times $𝑟$ and $𝑡$, divided by the time interval. Note that:

• $𝑢$ is a functional of $𝑣$: $𝑢=ℱ\left[𝑣\right]$
• As $𝑟\to 𝑡$: $\underset{𝑟\to 𝑡}{\lim }𝑢\left({𝑧}_{𝑡},𝑟,𝑡\right)=𝑣\left({𝑧}_{𝑡},𝑡\right)$
• The field $𝑢\left({𝑧}_{𝑡},𝑟,𝑡\right)$ depends on both time points $𝑟$ and $𝑡$

1.2. The MeanFlow Identity

1.2.1. Derivation

The fundamental relationship between average and instantaneous velocities emerges from differentiating the definition. Starting with:

$$\left(𝑡-𝑟\right)𝑢\left({𝑧}_{𝑡},𝑟,𝑡\right)={\int }_{𝑟}^{𝑡}𝑣\left({𝑧}_{𝜏},𝜏\right)𝑑𝜏$$

Differentiating both sides with respect to $𝑡$ (treating $𝑟$ as independent):

$$\frac{𝑑}{𝑑𝑡}\left[\left(𝑡-𝑟\right)𝑢\left({𝑧}_{𝑡},𝑟,𝑡\right)\right]=\frac{𝑑}{𝑑𝑡}{\int }_{𝑟}^{𝑡}𝑣\left({𝑧}_{𝜏},𝜏\right)𝑑𝜏$$

Using the product rule on the left and fundamental theorem of calculus on the right:

$$𝑢\left({𝑧}_{𝑡},𝑟,𝑡\right)+\left(𝑡-𝑟\right)\frac{𝑑𝑢}{𝑑𝑡}=𝑣\left({𝑧}_{𝑡},𝑡\right)$$

This gives us the MeanFlow Identity:

1.2.2. Computing the Total Derivative

The total derivative $\frac{𝑑𝑢}{𝑑𝑡}$ expands as:

$$\frac{𝑑𝑢}{𝑑𝑡}=\frac{𝑑{𝑧}_{𝑡}}{𝑑𝑡}{\partial }_{𝑧}𝑢+\frac{𝑑𝑟}{𝑑𝑡}{\partial }_{𝑟}𝑢+\frac{𝑑𝑡}{𝑑𝑡}{\partial }_{𝑡}𝑢$$

Since $\frac{𝑑{𝑧}_{𝑡}}{𝑑𝑡}=𝑣\left({𝑧}_{𝑡},𝑡\right)$, $\frac{𝑑𝑟}{𝑑𝑡}=0$, and $\frac{𝑑𝑡}{𝑑𝑡}=1$:

$$\frac{𝑑𝑢}{𝑑𝑡}=𝑣\left({𝑧}_{𝑡},𝑡\right){\partial }_{𝑧}𝑢+{\partial }_{𝑡}𝑢$$

This can be efficiently computed using Jacobian-Vector Products (JVP) with tangent vector $\left[𝑣,0,1\right]$.

1.3. Training Procedure

1.3.1. Loss Function

We parameterize the average velocity with a neural network ${𝑢}_{𝜃}\left({𝑧}_{𝑡},𝑟,𝑡\right)$ and encourage it to satisfy the MeanFlow Identity:

$$ℒ\left(𝜃\right)=𝔼\left[{\Vert {𝑢}_{𝜃}\left({𝑧}_{𝑡},𝑟,𝑡\right)-\text{sg}\left({𝑢}^{\text{ tgt}}\right)\Vert }^2\right]$$

where the target is:

$${𝑢}^{\text{tgt }}={𝑣}_{𝑡}-\left(𝑡-𝑟\right)\left({𝑣}_{𝑡}{\partial }_{𝑧}{𝑢}_{𝜃}+{\partial }_{𝑡}{𝑢}_{𝜃}\right)$$

Here:

• ${𝑣}_{𝑡}=𝜀-𝑥$ is the conditional velocity from Flow Matching
• $\text{sg}(\cdot )$ denotes stop-gradient to avoid higher-order derivatives
• The JVP computation $\left({𝑣}_{𝑡}{\partial }_{𝑧}{𝑢}_{𝜃}+{\partial }_{𝑡}{𝑢}_{𝜃}\right)$ is efficient

1.4. Classifier-Free Guidance Extension

1.4.1. Motivation

Standard CFG requires two function evaluations: ${𝑣}^{\text{cfg }}=𝜔𝑣\left(\cdot |𝑐\right)+\left(1-𝜔\right)𝑣(\cdot )$. For one-step generation, this doubles the Number of Function Evaluations (NFE), defeating the purpose.

1.4.2. Solution: CFG in the Ground Truth

MeanFlow naturally incorporates CFG into the ground-truth field itself:

$${𝑣}^{\text{cfg}}\left({𝑧}_{𝑡},𝑡|𝑐\right)=𝜔𝑣\left({𝑧}_{𝑡},𝑡|𝑐\right)+\left(1-𝜔\right)𝑣\left({𝑧}_{𝑡},𝑡\right)$$

The corresponding average velocity ${𝑢}^{\text{cfg}}$ satisfies its own MeanFlow Identity:

$${𝑢}^{\text{cfg}}\left({𝑧}_{𝑡},𝑟,𝑡|𝑐\right)={𝑣}^{\text{ cfg}}\left({𝑧}_{𝑡},𝑡|𝑐\right)-\left(𝑡-𝑟\right)\frac{𝑑{𝑢}^{\text{ cfg}}}{𝑑𝑡}$$

1.4.3. Training with CFG

The modified target becomes:

$${\widetilde{𝑣}}_{𝑡}=𝜔{𝑣}_{𝑡}+\left(1-𝜔\right){𝑢}_{𝜃}^{\text{ cfg}}\left({𝑧}_{𝑡},𝑡,𝑡\right)$$

This allows the network to directly model the CFG-enhanced field while maintaining 1-NFE sampling.

References

1. Mean Flows for One-step Generative Modeling