Downloads · 30 days
0
scbe-tokenizer-sacred-tongues-qwen25-coder-7b
issdandavis/scbe-tokenizer-sacred-tongues-qwen25-coder-7b
scbe-tokenizer-sacred-tongues-qwen25-coder-7b is a machine learning model from issdandavis. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Status: experimental. Research artifact, not a production candidate. Canonical model: scbe-coding-agent-vtc-qwen15-v1-gguf.
Access
Public
Updated Jul 29, 2026
Repo size
11.4 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json11.4 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- May 6, 2026
- Updated
- Jul 29, 2026
- SHA
- bd933362
Model card
From the Hugging Face model README
<!-- scbe-status -->
Status: experimental. Research artifact, not a production candidate. Canonical model: scbe-coding-agent-vtc-qwen15-v1-gguf.
Sacred Tongues tokenizer - Qwen2.5-Coder-7B
This is a tokenizer only. There are no model weights in this repo. Load it
with AutoTokenizer, not AutoModel.
Numbers below are from sacred_tongues_extension_report.json in this repo.
Twelve tongue names were added to the vocabulary, lowercase and capitalised:
kor'aelin avali runethic cassisivadan umbroth draumric.
- Vocab size: 151,665 -> 151,677 (12 requested, 12 added)
- Effect: every one of the twelve went from 2-4 pieces to 1 piece
| token | pieces before | pieces after |
|---|---|---|
| kor'aelin | 4 | 1 |
| cassisivadan | 4 | 1 |
| runethic | 3 | 1 |
| umbroth | 3 | 1 |
| draumric | 3 | 1 |
| avali | 2 | 1 |
That is the whole claim: the six tongue names are now atomic instead of being split across several sub-word pieces. No downstream model quality is claimed.
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("issdandavis/scbe-tokenizer-sacred-tongues-qwen25-coder-7b")
- License
- apache-2.0
- Created
- May 6, 2026
- Updated
- Jul 29, 2026