We want to sample from $𝑝\left(𝑥\right)$, but how? We usually have four ways:

Paradigm	Idea
Directly model $𝑝\left(𝑥\right)$	Explicit density: $𝑝\left(𝑥\right)=\frac{𝑞\left(𝑥\right)}{𝑍},𝑍=\int 𝑞\left(𝑥\right)𝑑𝑥$ Or tractable forms: $𝑝\left(𝑥\right)={\prod }_{𝑖}𝑝\left({𝑥}_{𝑖}|{𝑥}_{<𝑖}\right)$
Latent Variable $𝑝\left(𝑥|𝑧\right)$	Marginalization: $𝑝\left(𝑥\right)=\int 𝑝\left(𝑥|𝑧\right)𝑝\left(𝑧\right)𝑑𝑧$ Variational bound: $\log 𝑝\left(𝑥\right)\ge \text{ ELBO}$
Implicit Generation $𝐺\left(𝑧\right)\to 𝑥$	Pushforward measure: ${𝑝}_{𝑔}\left(𝑥\right)={\left({𝐺}_{𝜃}\right)}_{\#}𝑝\left(𝑧\right)$ No explicit density
Score-Based ${\nabla }_{𝑥}\log 𝑝\left(𝑥\right)$	Score matching: Learn ${𝑠}_{𝜃}\left(𝑥\right)\approx {\nabla }_{𝑥}\log 𝑝\left(𝑥\right)$ Reverse diffusion process

1. Continuous Generative Models

• Discrete Generative Models

  • Pros:

    • Efficient Inference: They can be very fast, often requiring only one pass of the transformer to generate a sequence.

  • Cons:

    • Quality Issues: Discrete tokens suffer from a "quality issue" due to high data compression, which results in the loss of fine details. Reconstructed images can look significantly different up close from the original.
    • Fundamental Compression Flaw: To be manageable, a sequence of discrete tokens must compress information far more than a continuous representation, which is a fundamental limitation.

• Continuous Generative Models

  • Pros:

    • High-Quality Samples: They generally offer much better reconstruction than discrete models.

  • Cons:

    • Speed Issues: Continuous models, particularly diffusion, have a "speed issue" because they require many iterative steps to generate a sample. This multi-pass process makes inference slow and computationally demanding.

Do inference-time scaling benefit generative pre-training algorithms? Maybe.

References

1. New Pre-training Paradigms from a Inference-First Perspective
2. Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithms