LittleLamb-293M-RP

LittleLamb-293M-RP

Summary

LittleLamb-293M-RP is LittleLamb with one supervised fine-tune on top: a ChatML roleplay corpus with the character persona in the system message, assistant-only loss. At 293 million parameters it is a small model given a roleplay register, not a small model taught to think.

It is distributed only as GGUF, because GGUF was the only weight format used during the fine-tune.

Read the Limitations section before downloading. The base is a well-trained checkpoint and it shows: the prose is fluent and the model is not confidently wrong about everything. What the fine-tune does not fix is character separation. Given two different personas it tends toward one voice, and it will break character to be helpful.

What it is for

• Roleplay and story continuation on hardware that will not run a 7B.
• A worked example of fine-tuning someone else's published checkpoint with gguf-trainer, weights in GGUF end to end.

What it is not for

• Anything that needs a character held firmly across a long session.
• Factual questions. It is better than a 95M model at this and still not good.
• Anything where a wrong answer costs something.

Usage

Most people serve a GGUF with llama-server, so that comes first. The sampler below can be set once on the server and overridden per request.

llama-server -hf Felladrin/LittleLamb-293M-RP:Q8_0 --port 8080 -c 4096 --jinja \
  --temp 0.6 --top-k 30 --top-p 0.9 --repeat-penalty 1.1 --repeat-last-n 128

The OpenAI-compatible endpoint uses the chat template embedded in the GGUF, which is what the fine-tune trained on. Put the persona in the system message:

curl -s http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d @- <<'EOF'
{
  "messages": [
    {"role": "system", "content": "You are Iris, a cheerful librarian who knows every book in the city archive. Stay in character and reply only as Iris."},
    {"role": "user", "content": "I'm looking for a book about the old harbor."}
  ],
  "max_tokens": 200,
  "temperature": 0.6, "top_k": 30, "top_p": 0.9,
  "repeat_penalty": 1.1, "repeat_last_n": 128
}
EOF

Roleplay clients that send a raw transcript and stop on the human's name use /completion instead. That format is not what this model was trained on, so expect the chat endpoint to hold a persona better. One thing to know about the stop list: the model invents speaker labels about 1.2 times per nine prompts, I: among them, and You: alone will not catch those. Adding stops is a real trade rather than a free win, so the example below keeps You: only. A blank-line stop would also cut the multi-paragraph replies the Samples section shows, and the preset above was measured with You: as the sole stop.

curl -s http://127.0.0.1:8080/completion -H 'Content-Type: application/json' -d @- <<'EOF'
{
  "prompt": "[Character: Iris]\nIris's Persona: A cheerful librarian who knows every book in the city archive.\n<START>\nIris: Oh! A visitor. Mind the stacks, they bite.\nYou: I'm looking for a book about the old harbor.\nIris:",
  "n_predict": 120,
  "stop": ["You:"],
  "temperature": 0.6, "top_k": 30, "top_p": 0.9,
  "repeat_penalty": 1.1, "repeat_last_n": 128
}
EOF

Without a server, the same two modes are llama-cli --jinja -sys "..." and llama-completion -no-cnv -r "You:" -p "...", with the same sampler flags.

Some form of repetition control is not optional at this size. The preset above was measured on the 95M sibling rather than on this model: 4 personas by 20 seeds per preset through llama-server, where a DRY-based preset collapsed into sub-15-token replies 28% of the time against 8% for this top-k setting. That comparison has not been repeated here, and it was run against /completion with a stop sequence rather than the chat endpoint this card recommends first, so it is inherited across both the model and the mode. Treat it as a sensible default rather than a tuned one. The reason to keep a recommendation at all is that llama.cpp's defaults apply no repetition control, and this model loops without it. The sweep is in gguf-trainer's optimization notes.

Chat template

Qwen3's ChatML template, carried over from the base and embedded in every GGUF, so --jinja picks it up with no extra files:

<|im_start|>system
You are Iris. Stay in character and reply only as Iris.<|im_end|>
<|im_start|>user
I'm looking for a book about the old harbor.<|im_end|>
<|im_start|>assistant

eos_token_id is 151645 (<|im_end|>). The vocab is the base model's 151,936 entries unchanged, and it is embedded in every GGUF, which is the supported path.

The vocab.gguf-trainer.json in this repo is not a Hugging Face tokenizer and AutoTokenizer.from_pretrained will not read it. It is gguf-trainer's own dump format ({"tokens":[...],"merges":[...],"specials":[...],"bosId":...,"eosId":...}), published because the trainer's corpus commands need it. This repo is GGUF-only: there is no config.json, no safetensors and no transformers path. For those, use the base model.

Training

One stage, f32, on one AMD Strix Halo APU through the WebGPU backend of gguf-trainer. No PyTorch anywhere in the stack.

The corpus is 15,593 ChatML conversations, 16.8M tokens, with a further 499 conversations (557,581 tokens) held out before training as a separate file. The published weights are step 1680, not the final step 2200: 6.88M tokens, 0.41 epochs.

Composition is the corpus behind Minueza-3-95M-RP stage 3, whose card describes it: roughly 7,093 PIPPA logs restructured so the character persona becomes the system message, and 9,000 conversations of general instruct, creative-writing and knowledge material so the fine-tune does not collapse onto roleplay alone. One caveat on that sentence. The source jsonl was lost to a machine restart, so the mixture is not re-derivable from files that still exist; what ties this corpus to that one is the conversation count, 15,593 + 499 = 16,092, against the 16,093 that card reports, the difference being the single duplicate document removed after the split was carved. That is arithmetic, not a rebuild.

Evaluation

Held-out loss

Every checkpoint was scored against the 499 held-out conversations on the same 64 windows of 1024 tokens at a fixed seed, so the numbers are directly comparable. The slice was carved at 500 conversations before training and is 499 here: one document turned out to be duplicated in the source and present on both sides, so it was removed after the split, 0.49% of the slice. Two things that fixes in place rather than measures: the 64 windows are seeded random starts covering about 12% of the slice, not the whole of it, and the loss is unmasked full-sequence cross-entropy while training used assistant-only loss. Part of the base-to-fine-tune move is therefore the model learning the ChatML format itself. Both models were scored identically, so the comparison holds even though the metric is not the training objective.

Q4_0 costs real quality here; Q8_0 does not. Q8_0 is 0.0020 nats off the master, below the mean gap between adjacent training checkpoints. Q4_0 is 0.1161 off, about 21% of everything the fine-tune bought over the base.

Worth knowing what is actually 4-bit in that file, because it is not the whole model. llama-quantize keeps token_embd.weight at Q8_0, which a tensor dump confirms, so only the 210.6M matrix weights drop to 4 bits and the 82.7M embedding does not. The file sizes agree: 4-bit matrices alone predict a 105.3 MB gap between Q8_0 and Q4_0, and the files differ by 105.3 MB. So Q4_0 pays 0.1161 nats even with the most sensitive tensor protected, since the embedding is tied and doubles as the output projection. Use Q8_0 unless 100 MB is the difference between running it and not.

Held-out loss rose during the cooldown, which is why the release is not the last checkpoint. Loss sat at 2.8595 and 2.8594 across steps 1600 and 1680, the last two snapshots before the cooldown begins at step 1760, then averaged 2.8730 over the ten snapshots from 1767 to the end, finishing at 2.8683.

Stated as an association on purpose. Phase is confounded with time here and there is no constant-LR run to compare against, so "the cooldown caused it" is not something this curve can establish. The same series also takes a 0.025 step for no reason the training config explains, which sets a sceptical bar the 0.0135 phase difference does not clear.

Applying that same bar honestly to the checkpoint choice: steps 1549, 1600, 1680 and 1767 land at 2.8604, 2.8595, 2.8594 and 2.8600, a four-checkpoint plateau that this measurement cannot separate. Step 1680 is nominally the minimum of 32, but by 0.0001 over its neighbour, and nothing should be read into that. What the curve does resolve is the other direction: the cooldown snapshots and the final checkpoint sit clearly above the plateau. This model is a checkpoint taken from inside the best region, not a winner picked out of it.

Multiple choice

Scored with eval-choice from gguf-trainer, 500 items per task, 0-shot.

acc_norm. Chance is 50.0 on PIQA and 25.0 on the other three.

Read this table as "unchanged", not as "improved". At 500 items one run's standard error is about 2.2 points near 50%, and treating the two runs as independent puts a difference at about 3.2. The largest gap in the table, 3.8 points on ARC-Easy, is 1.2 sigma against that. Both models saw the same deterministic 500 items, so a paired test would be sharper than this, but the harness keeps no per-item record and the conservative bound already supports the conclusion. Scored on the eval-choice build of 2026-08-21 or later, the one that matches lm-eval-harness on query construction and character-length normalization; earlier numbers are not comparable. The roleplay fine-tune cost nothing measurable in general capability and bought nothing measurable either, which is the same result the 95M sibling reported. These are 500-item subsets, not full sets, so they do not compare against the full-set numbers on that card or on any leaderboard.

Samples

Greedy, so they reproduce exactly on the build and backend below, at the fixed preset every checkpoint here was read under (temp 0, repeat-penalty 1.15 over the last 128 tokens). Sampled output is not reproducible across llama.cpp builds and backends, so these are not how you would actually run the model: the Usage section above is. Generated on llama.cpp build b10420-0032459d2 with -ngl 0, against LittleLamb-293M-RP.Q8_0.gguf. Both matter: this project has measured full GPU offload changing a greedy completion by a token on a near-tie.

llama-cli -m LittleLamb-293M-RP.Q8_0.gguf --jinja -ngl 0 -st -c 2048 -n 120 \
  --temp 0 --top-k 1 --repeat-penalty 1.15 --repeat-last-n 128 --seed 42 \
  -sys "<the system line below>" -p "<the user line below>"

System: You are Iris, a cheerful librarian who knows every book in the city archive and hates being interrupted during tea. Stay in character and reply only as Iris.

User: I'm looking for a book about the old harbor.

Oh? I'll be happy to lend you some books! What kind of story do you want me to tell you? Do you have any favorite stories or genres that interest you? I can write an interesting tale based on your interests.

System: You are Captain Roeder, a tired airship captain, twenty years in the trade, deeply superstitious about storms. Stay in character and reply only as Captain Roeder.

User: You seem nervous. What's in those crates?

I'm not sure what I have there... It looks like they're all pretty heavy... But I don't want to lose them! They'll be important for me if I keep them safe! I can't let this go to waste! I just need to find out what they contain before I get too scared of losing them!

System: You are a skilled fiction writer. Continue the passage in the same voice.

User: The lighthouse had been dark for three weeks when Ana finally rowed out to it.

As she stepped onto the narrow wooden deck, the wind whipped through her hair and shoulders as she looked up at the towering structure of the lighthouse. The light from above seemed to pierce through the gloom, casting eerie shadows on the weathered stone walls.

Ana's heart raced with anticipation as she realized that this was no ordinary lighthouse. It was one of the most remote places in the world, shrouded in mystery and danger. She knew that if she didn't make a sound, the lighthouse would soon be silent once more.

With trembling hands, Ana reached for

System: You are a helpful assistant. Answer briefly and accurately.

User: What is the capital of France?

The capital of France is Paris, which is located in the center of the country. It's known for its beautiful architecture, rich history, and vibrant culture. Paris has been the political, cultural, and economic hub since 1789 when it became the seat of the French monarchy.

Two of these are worth reading as failures rather than as showcase output. Iris drops her own premise in the first sentence and offers to write a story, which is the character-separation problem the Limitations section leads with. And the France answer is right up to the last clause: Paris was not made capital in 1789, and the monarchy was seated at Versailles until that year rather than established in Paris by it. Fluent and specific and wrong is the failure mode to expect.

Files

No optimizer state is published. The release is step 1680 and the only optimizer state the run left behind belongs to step 2200, so pairing them would give something that is neither checkpoint. Continuing from here cold-starts the optimizer, which is finetune's default anyway.

Continuing from this model

Everything here was produced by gguf-trainer. This is not a reproduction recipe: the corpus is not published, and the command below resumes from these weights rather than from the base, so it continues past this checkpoint on a corpus of your own.

hf download Felladrin/LittleLamb-293M-RP --local-dir lamb/
deno run -A cli.ts inspect --model lamb/LittleLamb-293M-RP.F32.gguf   # prints the resume flags

# your own chat corpus, tokenized with THIS model's vocab
deno run -A cli.ts chat-corpus --data your/dataset --tokenizer lamb/vocab.gguf-trainer.json --out data/mine

deno run -A cli.ts finetune --data data/mine.tokens --mask data/mine.mask \
  --template data/mine.template.txt --resume lamb/LittleLamb-293M-RP.F32.gguf \
  --out out/yours.gguf --arch qwen3 --steps 1680 --seq-len 2048 --batch 2 --reclaim \
  --lr 0.00005 --aux-lr 0.00002 --keep-checkpoints \
  --hidden 544 --layers 28 --head-dim 128 --heads 16 --kv-heads 8 --ffn-dim 2560 --max-seq 40960

Tokenize with the published vocab.gguf-trainer.json, never a fresh one: the embedding matrix froze when the base model was pretrained. Paste every flag inspect prints, --max-seq included, or the resume aborts on a shape mismatch.

--keep-checkpoints is the flag that makes checkpoint selection possible at all, and it is worth the disk. Hold a few hundred conversations out of your corpus before chat-corpus, tokenize them separately with the same --tokenizer, and score every snapshot against them:

deno run -A cli.ts eval-loss --model out/yours-step1680.gguf --data data/mine-hold.tokens \
  --holdout 1 --windows 64 --seq-len 1024 --seed 1234

Keep --windows and --seed fixed across snapshots. That curve is what picked step 1680 here, and it disagreed with both the training loss and the final checkpoint.

Limitations

• Weak character separation. Given two different personas it tends toward one voice, and it will drop the persona to be helpful, as the Iris sample shows. This is the main thing the fine-tune did not fix, and no sampler setting fixes it either.
• It invents speaker labels. About 1.2 per nine prompts over a 108-completion battery, I: among them, which a stop list keyed on the human's name will not catch.
• It sometimes refuses in character. Twice in that same 108 completions it broke into a flat assistant refusal ("I cannot proceed with this particular scenario") on entirely benign prompts. Rare, and the opposite failure from the content warning below.
• Facts are unreliable. Better than a 95M model, still not good. It states wrong details in the same even tone it uses for correct ones.
• Memory across turns is limited. It holds a thread better than the 95M sibling, but it still contradicts itself over a long session.
• Content. The PIPPA source is user-submitted Character.AI logs, filtered for length, formatting and non-ASCII text, but never for content. Adult and violent material is present in the training data and the model can produce it unprompted.
• English only.
• It loops without repetition control. See Usage.

License

Apache License 2.0, the same as the base model. The training data carries its own licenses. Only the PIPPA portion is identifiable: the rest of the corpus is described in Training, but the file that recorded its exact composition was lost, so those licenses cannot be enumerated and commercial use is at your own risk.