Downloads · 30 days
0
qwen38-flash-next-tokenizer
simonepstein/qwen38-flash-next-tokenizer
qwen38-flash-next-tokenizer is a machine learning model from simonepstein. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Access
Public
Updated Sep 21, 2026
Repo size
12.8 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json12.8 MB · 100%
At a glance
- Access
- Public
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026
- SHA
- 80b7acf6
Model card
From the Hugging Face model README
Qwen3.8-Flash-Next tokenizer
tokenizer.json copied unmodified from Qwen/Qwen3.8-Flash-Next-FP8.
Exists so a deployment can point at a 12.8 MB repo rather than pull a 120 GB one for a single file. Used with dgpp's engine.tokenizer from when serving a repackaged Flash-Next checkpoint whose own tokenizer.json carries a pre-tokenizer that differs from the release its vocabulary comes from.
No changes to the file. Qwen license applies.
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026