apex-vq-1024

Apex VQ 1024

An image tokenizer. Turns a picture into 1,024 discrete tokens — apexes — and back into an approximation of it. Model 263701, the first of the Apex family.

Named for the solar apex, the point on the sky the Sun is travelling toward. The codebook is a fixed set of reference directions and every patch is matched to the nearest one, which is the same idea.

from imagetok import PatchVQTokenizer
from PIL import Image

tok = PatchVQTokenizer.load("imagetok_1024.npz")
ids = tok.encode(Image.open("photo.jpg"))   # 1024 ints, each 0..1023
back = tok.decode(ids)                      # a PIL image

It is lossy, and that is not a bug to be fixed

A text tokenizer is exactly invertible — byte-level BPE returns the bytes it was given, and anything less is broken. An image tokenizer cannot do that and nothing will make it: a photograph is millions of continuous values and a token is one integer out of a thousand. The mapping discards information by design.

So fidelity is reported as a quantity, not a pass mark. There is no pass mark, and whether coarse is acceptable depends on what the tokens are for.

Measured on photographs it never saw

What the numbers say about where the loss comes from

Quadrupling the codebook from 256 to 1024 buys 0.9 dB (22.5 → 23.4). Halving the patch size buys 2.3 dB and costs four times the tokens:

So the loss is dominated by the patch grid, not the codebook. An 8×8 patch cannot represent an edge crossing it at an angle however many codes exist, which is the visible failure — straight edges go blocky while colour and layout survive.

That also bounds anything built on top: a model that generates these tokens can never produce an image better than this tokenizer can reconstruct. If you are using this as the front end of a generative or editing model, 23.4 dB is your ceiling, and a learned convolutional VQ-VAE at the same token count would raise it — the encoder can place information anywhere in the patch instead of matching against a fixed dictionary.

Method

k-means over patches drawn from the training corpus, k-means++ seeded so that a corpus of mostly-sky does not put every code in the sky. Nearest code wins and its index is the token. This is what a VQ-VAE does with a neural encoder in front; without one it stays inspectable — every token is a patch you can look at, so codebook collapse is visible rather than theoretical.

Trained on 272 CC-licensed photographs from Wikimedia Commons, held out 48 for the numbers above. A larger and more varied corpus would move these figures; they are reported on what was actually used.

Licence

Apache 2.0. imagetok.py and metric.py are included so the numbers can be reproduced rather than taken on trust.