1. KARL (Kolmogorov-Approximating Representation Learning)

Traditional computer vision models rely on fixed-length representations for all images, regardless of their inherent complexity. This approach contradicts fundamental principles from Algorithmic Information Theory (AIT), which suggests that intelligent systems should compress data into their shortest possible programs. Simple images with repetitive patterns should require fewer tokens than complex scenes with intricate details, yet most current methods allocate identical computational resources to both.

The system uses a Perceiver-inspired architecture with three main components:

• Latent Distillation Encoder: Processes 2D image tokens alongside learnable 1D latent tokens, producing both token embeddings and halting probabilities
• Adaptive Masking: Uses learned halting probabilities to determine which tokens are essential for reconstruction
• Cross-Attention Decoder: Reconstructs images using only active (non-halted) tokens

The training process alternates between two phases within each iteration:

Phase 1: Estimate Image Complexity (EIC)

• Input: Image $𝑥$ and random token budget $𝑇$
• Target: Near-lossless reconstruction ($𝜀=0$)
• Output: Empirical reconstruction error ${𝜀}_0$

The loss function for this phase is:

$${ℒ}_{\text{EIC }}={ℒ}_{\text{ recon }}\left({\widehat{𝑥}}_{𝑇},𝑥\right)+𝛽{ℒ}_{\text{ quant }}\left({𝑧}_{𝑇}\right)$$

Phase 2: Learn to Tokenize Complexity (LTC)

• Input: Same image $𝑥$, expanded token budget $𝑇+\mathrm{\Delta }𝑇$, and target quality ${𝜀}_0$
• Target: Match the reconstruction quality ${𝜀}_0$ using only $𝑇$ tokens
• Output: Token embeddings and halting probabilities

The total loss includes reconstruction, quantization, and halting components:

$${ℒ}_{\text{LTC }}={ℒ}_{\text{ recon }}\left({\widehat{𝑥}}_{𝑇+\mathrm{\Delta }𝑇},𝑥\right)+𝛽{ℒ}_{\text{ quant }}\left({𝑧}_{𝑇+\mathrm{\Delta }𝑇}\right)+𝜆{ℒ}_{\text{ halt }}\left(𝜔\right)$$

The halting loss encourages the model to preserve the first $𝑇$ tokens while discarding the additional $\mathrm{\Delta }𝑇$ tokens:

$${ℒ}_{\text{halt }}\left(𝜔\right)=\text{BCE}\left({𝜔}_{0:𝑇},0\right)+\text{BCE}\left({𝜔}_{𝑇:𝑇+\mathrm{\Delta }𝑇},1\right)$$