Behemoth-T1-123B-FP8

<div align="center"> <img src="https://huggingface.co/tacodevs/Behemoth-T1-123B/resolve/main/images/01_header_main.png" alt="Behemoth-T1" width="100%" /> <h1>🌴 Behemoth-T1-123B-FP8 🌴</h1> <p><i>The party where literary craft meets unhinged creative writing — production sweet spot.</i></p> <p> <a href="https://huggingface.co/tacodevs/Behemoth-T1-123B"><img src="https://img.shields.io/badge/BF16-tacodevs%2FBehemoth--T1--123B-ff7eb6?style=for-the-badge" alt="BF16"/></a> <a href="https://huggingface.co/tacodevs/Behemoth-T1-123B-FP8"><img src="https://img.shields.io/badge/FP8-Behemoth--T1--123B--FP8-7eddff?style=for-the-badge" alt="FP8"/></a> <a href="https://huggingface.co/tacodevs/Behemoth-T1-123B-GPTQ"><img src="https://img.shields.io/badge/GPTQ--W4A16-Behemoth--T1--123B--GPTQ-bdff7e?style=for-the-badge" alt="GPTQ"/></a> </p> </div>

☀️ The pitch

This is the FP8 W8A8 dynamic quantized version of tacodevs/Behemoth-T1-123B — a 123B Mistral Large roleplay model that thinks like a literary author before it writes like a storyteller.

FP8 is the production sweet spot: half the VRAM of BF16, ~99% of the quality (essentially lossless), runs cleanly on Hopper GPUs (H100, H200) with native FP8 acceleration. This is the variant most users should pick.

For the full pitch, training details, and the philosophy behind T1, see the BF16 model card.

⚡ This variant

🎤 How to use

T1 expects a prefilled <think> block to enter literary thinking mode. Use the same 7 prefill phrases as the BF16 model:

import openai

client = openai.OpenAI(base_url="http://localhost:8000/v1", api_key="-")

PREFILLS = {
    "analytical": "Ok i need to think about how to respond — what does the character feel right now, what from their experience is relevant, what do they value, and what are they trying to achieve, so",
    "creative":   "Ok i need to think as a creative writer — what twist would surprise here? Let me find an engaging new direction nobody saw coming, so",
    "unhinged":   "Ok i need to think as an unhinged author — raw, explicit, intense, fully in character with no holding back, so",
}

response = client.chat.completions.create(
    model="tacodevs/Behemoth-T1-123B-FP8",
    messages=[
        {"role": "system", "content": CHARACTER_CARD},
        *conversation_history,
        {"role": "user", "content": user_message},
        {"role": "assistant", "content": f"<think>\n{PREFILLS['creative']}\n"},
    ],
    extra_body={
        "continue_final_message": True,
        "add_generation_prompt": False,
    },
    temperature=0.6,
    max_tokens=2048,
    stop=["[INST]", "</s>"],
)

🚀 Serving with vLLM

vllm serve tacodevs/Behemoth-T1-123B-FP8 \
    --tokenizer-mode auto \
    --tensor-parallel-size 2 \
    --max-model-len 8192 \
    --kv-cache-dtype fp8

Important: use --tokenizer-mode auto, not mistral — mistral_common mode silently mis-templates merged-LoRA checkpoints.

Recommended hardware:

• 2× H100 80GB — comfortable, ~62GB per GPU
• 2× H200 144GB — luxury, plenty of headroom for 32k+ context
• 1× H100 NVL 188GB — single-card option

FP8 W8A8 uses native FP8 tensor cores on Hopper GPUs for both weights and activations, so this variant is also faster than BF16 in practice, not just smaller.

✅ Quality notes

FP8 W8A8 dynamic quantization is essentially lossless for 100B+ models:

• ✅ Stream-of-consciousness thinking shape — preserved
• ✅ Detail surfacing from character cards — preserved
• ✅ Word-for-word output similarity to BF16 — typically 95%+ token overlap on greedy sampling
• ✅ Beats base R1 in side-by-side — preserved
• ✅ Production reliability — same recipe Mistral officially uses for Mistral Large 3 FP8

If you need the absolute reference quality and have 4× 80 GB GPUs, use the BF16 reference. For most production use cases, FP8 is the right pick.

🛠️ Training details (from base T1)

T1 is a LoRA distillation of Claude Opus 4.5 literary thinking onto tacodevs/Behemoth-X-R1-123B (itself an SCE merge of Behemoth-X creative writing + Behemoth-R1 reasoning).

The LoRA only learns the shape of literary thinking. The base model's RP prose engine receives zero gradient updates — the underlying creative writing voice is structurally preserved.

📜 Citation

@misc{behemoth-t1-2026,
  title  = {Behemoth-T1-123B: Literary Thinking Distillation for RP},
  author = {tacodevs},
  year   = {2026},
  url    = {https://huggingface.co/tacodevs/Behemoth-T1-123B},
}

<div align="center">

<i>The party doesn't end. We just go to bed.</i>

</div>