Behemoth-T1-123B-GPTQ

<div align="center"> <img src="https://huggingface.co/tacodevs/Behemoth-T1-123B/resolve/main/images/01_header_main.png" alt="Behemoth-T1" width="100%" /> <h1>🌴 Behemoth-T1-123B-GPTQ 🌴</h1> <p><i>The party where literary craft meets unhinged creative writing — now in 4-bit.</i></p> <p> <a href="https://huggingface.co/tacodevs/Behemoth-T1-123B"><img src="https://img.shields.io/badge/BF16-tacodevs%2FBehemoth--T1--123B-ff7eb6?style=for-the-badge" alt="BF16"/></a> <a href="https://huggingface.co/tacodevs/Behemoth-T1-123B-FP8"><img src="https://img.shields.io/badge/FP8-Behemoth--T1--123B--FP8-7eddff?style=for-the-badge" alt="FP8"/></a> <a href="https://huggingface.co/tacodevs/Behemoth-T1-123B-GPTQ"><img src="https://img.shields.io/badge/GPTQ--W4A16-Behemoth--T1--123B--GPTQ-bdff7e?style=for-the-badge" alt="GPTQ"/></a> </p> </div>

☀️ The pitch

This is the W4A16 GPTQ-quantized version of tacodevs/Behemoth-T1-123B — a 123B Mistral Large roleplay model that thinks like a literary author before it writes like a storyteller.

GPTQ is for single-GPU users. The full model fits on a single 80 GB or 96 GB GPU. Quality is ~95-97% of the BF16 reference — virtually indistinguishable from full precision in normal use.

For the full pitch, training details, and the philosophy behind T1, see the BF16 model card.

⚡ This variant

🎤 How to use

T1 expects a prefilled <think> block to enter literary thinking mode. Use the same 7 prefill phrases as the BF16 model:

import openai

client = openai.OpenAI(base_url="http://localhost:8000/v1", api_key="-")

PREFILLS = {
    "analytical": "Ok i need to think about how to respond — what does the character feel right now, what from their experience is relevant, what do they value, and what are they trying to achieve, so",
    "creative":   "Ok i need to think as a creative writer — what twist would surprise here? Let me find an engaging new direction nobody saw coming, so",
    "unhinged":   "Ok i need to think as an unhinged author — raw, explicit, intense, fully in character with no holding back, so",
}

response = client.chat.completions.create(
    model="tacodevs/Behemoth-T1-123B-GPTQ",
    messages=[
        {"role": "system", "content": CHARACTER_CARD},
        *conversation_history,
        {"role": "user", "content": user_message},
        {"role": "assistant", "content": f"<think>\n{PREFILLS['creative']}\n"},
    ],
    extra_body={
        "continue_final_message": True,
        "add_generation_prompt": False,
    },
    temperature=0.6,
    max_tokens=2048,
    stop=["[INST]", "</s>"],
)

🚀 Serving with vLLM

vllm serve tacodevs/Behemoth-T1-123B-GPTQ \
    --tokenizer-mode auto \
    --max-model-len 8192 \
    --gpu-memory-utilization 0.9

Important: use --tokenizer-mode auto, not mistral — mistral_common mode silently mis-templates merged-LoRA checkpoints.

Single 80 GB H100, single 80 GB A100, or single 96 GB H100 NVL all fit comfortably.

🟡 Quality notes

W4A16 quantization has measurable but small impact on the literary thinking pattern T1 was trained for:

• ✅ Stream-of-consciousness thinking shape — preserved (encoded across many attention layers)
• ✅ Detail surfacing from character cards — preserved
• ✅ Beats base R1 in side-by-side — preserved (the gap is huge)
• 🟡 Specific word choices — may differ token-by-token from BF16
• 🟡 The "cleverest" inventive details — sometimes replaced with equivalent-quality alternatives

For most users, GPTQ T1 is indistinguishable from BF16 T1 in normal use. Only A/B testing with the same seed would expose the differences.

If you want maximum quality and have the VRAM, use the BF16 reference or FP8 W8A8 (~99% of BF16, fits on 2×80 GB).

🛠️ Training details (from base T1)

T1 is a LoRA distillation of Claude Opus 4.5 literary thinking onto tacodevs/Behemoth-X-R1-123B (itself an SCE merge of Behemoth-X creative writing + Behemoth-R1 reasoning).

The LoRA only learns the shape of literary thinking. The base model's RP prose engine receives zero gradient updates — the underlying creative writing voice is structurally preserved.

📜 Citation

@misc{behemoth-t1-2026,
  title  = {Behemoth-T1-123B: Literary Thinking Distillation for RP},
  author = {tacodevs},
  year   = {2026},
  url    = {https://huggingface.co/tacodevs/Behemoth-T1-123B},
}

<div align="center">

<i>The party doesn't end. We just go to bed.</i>

</div>