Downloads · 30 days
0
MUX
MisakiTaro0414/MUX
MUX is a text generation model from MisakiTaro0414. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Ayhan Suleymanzade<sup1</sup Halil Alperen Gozeten<sup2</sup Michael Bronstein<sup1,3</sup Ismail Ilkan Ceylan<sup1,4†</sup Jinwoo Kim<sup5†</sup
Access
Public
Updated May 11, 2026
Repo size
3.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin3.7 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- May 8, 2026
- Updated
- May 11, 2026
- SHA
- 10c77491
Try a prompt
Datasets
Model card
From the Hugging Face model README
MUX: Continuous Reasoning via Multiplexed Tokens
Ayhan Suleymanzade<sup>1</sup> Halil Alperen Gozeten<sup>2</sup> Michael Bronstein<sup>1,3</sup> Ismail Ilkan Ceylan<sup>1,4†</sup> Jinwoo Kim<sup>5†</sup>
<sup>1</sup>AITHYRA <sup>2</sup>University of Michigan <sup>3</sup>University of Oxford <sup>4</sup>TU Wien <sup>5</sup>KAIST <sup>†</sup>Equal advising
Overview
MUX compresses discrete chain-of-thought reasoning into continuous multiplexed tokens. Each latent token is trained to represent a weighted linear superposition of a span of discrete reasoning subwords via KL-divergence distillation.
Available Checkpoints
| File | Base Model | Dataset | Latent Tokens | GSM8K Accuracy |
|---|---|---|---|---|
mux-gpt2-gsm8k.bin | GPT-2 (124M) | GSM8K-AUG | 6 | 48.45% |
mux-llama1b-gsm8k.bin | LLaMA 3.2 1B-Instruct | GSM8K-AUG | 6 | 57.01% |
Usage
# Clone the MUX codebase
git clone https://github.com/<your-org>/MUX.git
cd MUX
# Download checkpoint
# Option 1: huggingface-cli
huggingface-cli download MisakiTaro0414/MUX mux-llama1b-gsm8k.bin --local-dir .
# Option 2: direct download
wget https://huggingface.co/MisakiTaro0414/MUX/resolve/main/mux-llama1b-gsm8k.bin
# Evaluate
CKPT_DIR=. bash scripts/eval_llama1b.sh
Training Details
- Training data: whynlp/gsm8k-aug (385K augmented GSM8K samples)
- Optimization: LoRA (r=128, alpha=32) with cosine LR schedule
- Loss: Superposition KL-divergence distillation + reference loss
- Hardware: NVIDIA H100 GPUs
Citation
@article{suleymanzade2025mux,
title = {MUX: Continuous Reasoning via Multiplexed Tokens},
author = {Suleymanzade, Ayhan and Gozeten, Halil Alperen and Bronstein, Michael and Ceylan, {\.I}smail {\.I}lkan and Kim, Jinwoo},
journal = {arXiv preprint},
year = {2025}
}
License
Apache 2.0
- Task
- Text Generation
- License
- apache-2.0
- Languages
- en
- Created
- May 8, 2026
- Updated
- May 11, 2026