Downloads · 30 days
0
VQ-VAE-Coconut-Qwen3-4B
Chengheng/VQ-VAE-Coconut-Qwen3-4B
VQ-VAE-Coconut-Qwen3-4B is a machine learning model from Chengheng. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as mit.
VQ-VAE bottleneck trained on the frozen Stage-3 Coconut latent thoughts of Qwen3-4B, from the VQ-CoT: Discretising Latent Chain-of-Thought project (team RateLimit Achieved, EPFL CS-552). The language model is frozen;…
Access
Public
Updated Jun 10, 2026
Repo size
55.9 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt55.9 MB · 100%
At a glance
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Jun 10, 2026
- Updated
- Jun 10, 2026
- SHA
- b663ffde
Model card
From the Hugging Face model README
VQ-VAE on Coconut latent thoughts (Qwen3-4B)
VQ-VAE bottleneck trained on the frozen Stage-3 Coconut latent thoughts of Qwen3-4B, from the VQ-CoT: Discretising Latent Chain-of-Thought project (team RateLimit Achieved, EPFL CS-552). The language model is frozen; only this bottleneck is trained on the dumped latents (385,620 x 6 latent thought vectors, GSM8K).
- Checkpoint:
vq.pt - K=4096 codes, code dim 64, EMA 0.999, k-means init + AE warmup
- Result: inserting this bottleneck into the frozen latent loop costs ~0 GSM8K test accuracy.
- Library
- pytorch
- License
- mit
- Created
- Jun 10, 2026
- Updated
- Jun 10, 2026