Downloads · 30 days
20
8% of all-time downloads
byte-llm-pretrain-v1
zuhri025/byte-llm-pretrain-v1
byte-llm-pretrain-v1 is a machine learning model from zuhri025. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A small Qwen3-architecture causal LM (31.6M params) trained from scratch with a byte-level tokenizer — 256 raw UTF-8 byte tokens plus <|pad|/<|bos|/<|eos|/[S1]/[S2]/3 reserved slots (264 total vocab). Because any Unic…
All-time downloads
248
Public
Parameters
31.6M
9.8 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.pt6.8 GB · 67%
At a glance
- Model type
- qwen3
- Access
- Public
- Created
- Sep 2, 2026
- Updated
- Sep 7, 2026
- SHA
- 98139576
Model card
From the Hugging Face model README
byte-llm-pretrain-v1
A small Qwen3-architecture causal LM (31.6M params) trained
from scratch with a byte-level tokenizer — 256 raw UTF-8 byte tokens
plus <|pad|>/<|bos|>/<|eos|>/[S1]/[S2]/3 reserved slots
(264 total vocab). Because any Unicode string decomposes losslessly into
UTF-8 bytes, this tokenizer covers any language/script with no fitting
step and no <unk>. [S1]/[S2] are greedily matched as single tokens
whenever their literal string appears in the input text.
Trained on the targets column of
humairmunirawn/xp3x-urdu.
See training_config.json in this repo for the full architecture and
optimisation config, and the runs/ folder for TensorBoard logs.
Usage
from transformers import AutoModelForCausalLM
import torch
model = AutoModelForCausalLM.from_pretrained("zuhri025/byte-llm-pretrain-v1")
# tokenizer: see byte_tokenizer.py in this repo, or infer.py for a full
# generation script with sampling, streaming, and an interactive mode.
- Type
- qwen3
- Created
- Sep 2, 2026
- Updated
- Sep 7, 2026