Downloads · 30 days
23
22% of all-time downloads
TinyGPT-Enhanced
SmallAICreator/TinyGPT-Enhanced
TinyGPT-Enhanced is a machine learning model from SmallAICreator. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Full-parameter, KL-anchored persona SFT of TGPT-XL (84.067M). Runs on-device at ~56–63 tok/s on a Pixel 6a (F16 GGUF).
All-time downloads
104
Public
Repo size
170 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf170 MB · 100%
At a glance
- Access
- Public
- Created
- Jul 15, 2026
- Updated
- Jul 15, 2026
- SHA
- ada9d155
Model card
From the Hugging Face model README
TGPT-XL-Persona
Full-parameter, KL-anchored persona SFT of TGPT-XL (84.067M).
Runs on-device at ~56–63 tok/s on a Pixel 6a (F16 GGUF).
- Developer: SmallAICreator (Kaden)
- Base model: TGPT-XL (85M param, trained from scratch on FineWeb-Edu + SmolTalk, ~8B tokens, ChatML, 1024 ctx)
- Format: ChatML
- Context length: 1024
- File:
tinygpt-persona-sft-f16.gguf(~170MB, F16) - License: (fill in — e.g. Apache-2.0 / MIT)
What this is
A persona/identity fine-tune of TGPT-XL that gives the model a stable self-identity (name: TinyGPT, maker: UltraLabs) without breaking base knowledge, code behavior, or instruction-following.
The interesting part isn't the persona — it's the training method. This was a full-parameter fine-tune (all 84.067M weights unfrozen) with a KL-divergence anchor to a frozen copy of the base weights, which is what prevents catastrophic forgetting. A LoRA attempt (v1) failed; the full-FT + KL-anchor approach (v2) drifted ~150× less.
Training recipe
| Setting | Value |
|---|---|
| Method | Full-parameter SFT (all weights unfrozen) |
| Trainable params | 84.067M (deduped for tied embedding) |
| Init from | sft_4000.pt |
| Learning rate | 2e-5, OneCycle |
| Steps | 300 |
| Hardware / time | CPU, ~50 min |
| Export | .pt → HF LlamaForCausalLM → convert_hf_to_gguf.py → F16 GGUF |
Loss:
- Created
- Jul 15, 2026
- Updated
- Jul 15, 2026