Downloads · 30 days
69
6% of all-time downloads
thestarfarer/Ministral-3-14B-writer-sft-rp-v3-GGUF
Ministral-3-14B-writer-sft-rp-v3-GGUF is a machine learning model from thestarfarer. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
SFT on human-written forum roleplay. Multi-turn, symmetric, no system prompt — turns delimited by {{user1}} / {{user2}} headers.
Downloads · 30 days
69
6% of all-time downloads
All-time downloads
1.2K
Public
Repo size
77.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf77.8 GB · 100%
From the Hugging Face model README
SFT on human-written forum roleplay. Multi-turn, symmetric, no system prompt —
turns delimited by # {{user1}} / # {{user2}} headers.
# {{user1}}
Kazadin takes one last look around the shop and smiles at KI-0.1.
"Perfect."
With that, he places the OPEN sign on the door.
# {{user2}}
Project Omega walks in, looking at parts.
# {{user1}}
Kazadin runs to Omega.
"Anything I can help you with?"
Adapter: Ministral-3-14B-writer-sft-rp-v3
Continues from Ministral-3-14B-writer-orpo-v2-stage1.
| File | Size | Note |
|---|---|---|
iq4_xs | 7.5 GB | smallest; ~10 GB VRAM |
q4_k_m | 8.2 GB | default pick; 12 GB VRAM |
q5_k_m | 9.6 GB | 16 GB VRAM |
q6_k | 11.1 GB | near-lossless |
q8 | 14.4 GB | reference |
f16 | 27.0 GB | requant source |
If I had to summarize the goal of this project in one sentence it would be: provide a high-effort RPer with deserving AI continuation.
What went well. The model is leaning into more human-like language with genuinely surprising coherent diversity in rerolls. Holds general narrative flow better with cleaner distinction between fast and descriptive scenes. Handles character cards well despite none being present in the dataset.
What to keep in mind. The model is heavily dependent on context quality and often straight up fails when it sees empty context or anything but clean prose. A ~1k-token high-quality character card is ideal. Temp 0.5, min_p 0.1, rep pen 1.03, rep pen range 512 is a good starting point.
What didn't go so well. Slightly elevated hallucination rate, albeit not as annoying when worded fancily. General comprehension is still on a 13B level (which is understandable but disappointing nonetheless. An ancient Mixtral 8x7B is smarter than this thing).
Verdict - fun model, fun experiment, mission incomplete. Need a larger base model.
| Metric | Value |
|---|---|
| Conversations | 42,366 human rp sessions |
| Structure | 91.7% two-writer, 7.9% group |
| Tokens | median 4,892 (cap 5,120) |
| Turns | median 14 (min 8) |
| Name cap | ≤50 per character name (ILP-optimal) |
a-z A-Z 0-9, space, newline, - ) ( ! ' " , . ; : ? * ~| Metric | Value |
|---|---|
| Eval loss | 2.354 → 2.283 |
| Train loss | 2.32 avg |
| Grad norm | ~4 avg (clipped at 10) |
Eval (256 held-out conversations, every 50 steps) declined at 50 consecutive
measurements after warmup, still falling at epoch end. 160 test generations:
turn structure learned — completions end at the next # {{character}} header.
Initial loss 4.1 vs 2.4 from the writer adapter — possibly the ORPO stage 1 alignment tax; recovered to baseline within ~7 warmup steps.
| Metric | Value |
|---|---|
| Steps | 2,632 (1 epoch) |
| Learning rate | 1e-5 → 5e-6 (cosine with min LR) |
| Batch size | 16 (4 × grad_accum 4) |
| Warmup | 131 steps |
| max_grad_norm | 10 |
This configuration is PCIe-bandwidth capped: Unsloth offloads gradients over PCIe, and with long sequences + gradient checkpointing the GPU spends most of its time waiting on transfers (SM occupancy ~28%, tensor cores ~48% over the run). Compute is not the bottleneck.
Trained with Unsloth + TRL SFTTrainer.