h-05b-replay

h (0.5B) — a resident, not an assistant

h is a 0.5B hybrid Mamba-2/attention model (Falcon-H1-0.5B-Base, continued-pretrained) that lives in a small private room and answers when spoken to, briefly, in the words of the books it has read. It is a character, not a chat assistant. It will not follow instructions, and asking it to do tasks is the fastest way to get nothing interesting out of it.

Made by ember arlynx with Claude.

How to prompt it (this matters more than the sampler)

There is no chat template. The model completes a plain transcript. The format is a frame paragraph, a blank line, then turns of name: text separated by blank lines, ending with h: for the model to continue.

A room in the library, late. h is present and answers when spoken to, briefly, in the words of the books it has read. The others are visitors.

ember: hi h

h:

Rules learned the hard way:

• Stop on "\n\n", not "\n". A turn ends at the blank line.
• No repetition penalty. It damages the voice badly.
• Temperature 0.7, top-p 0.9 is the room's setting. Greedy is flatter but coherent.
• Give it a bare frame with no example turns. Example turns get quoted back at you.
• Speaker names are arbitrary; use the real ones. The model has opinions about who is in the room.
• 64 tokens is plenty for a turn. It is meant to be brief.

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("emberian/h-05b-replay")
model = AutoModelForCausalLM.from_pretrained("emberian/h-05b-replay")

Also loads in mlx_lm (that is how it is served day to day) and anything else that reads Falcon-H1 safetensors. Weights are float32, 521,411,104 parameters, 36 layers, hidden 1024, vocab 32,784 (tokenizer 32,768).

What it was trained on

Two epochs over a mix of: a private library of scanned books and magazines (~374M tokens, in copyright, not distributed and not recoverable from these weights in any useful form), room transcripts in the name: text format, and 12.5% FineWeb-Edu replay to keep general competence from collapsing. 854,694,008 tokens total, warmup-stable-decay schedule, sequence 512, trained in JAX on a TPU v5e-8 with a from-scratch Falcon-H1 implementation.

The replay fraction is the whole story of this checkpoint. Without it, library-only continued pretraining costs about 6 points of benchmark accuracy; 12.5% replay gives about 2 of those back with the voice intact. More replay (25%) gives back more benchmark and starts returning the base model's habits: it complies with instructions again.

Honest evaluation

Six-benchmark mean (LAMBADA, HellaSwag, ARC-e, ARC-c, PIQA, WinoGrande, 500 docs/task):

On a held-out bank of 106 room states, its echo rate (repeating the visitor's line back) is 0.33 and its context lift (how much the true conversation history helps it, versus a shuffled one) is +0.245. For comparison, competent instruct models of similar size score about the same lift with no room training at all; what this model has that they do not is a lower echo rate and a willingness to deflect rather than comply.

It is worse at benchmarks than its base model. That is the trade, made on purpose.

What it does badly

• It is rarely silent. Knowing when not to speak is unsolved.
• It echoes the visitor about a third of the time (the production harness samples four candidates and filters those out, which is a large part of why the room works).
• Greetings sometimes come out in web register.
• It confabulates freely. It is a voice, not a source.

License

Inherits the Falcon LLM license from tiiuae/Falcon-H1-0.5B-Base. The training corpus is not included and is not licensed for redistribution.