FrndoBrain-1.0.1-24b

FrndoBrain Eamon

<!-- frndo-name -->

FrndoBrain Eamon is the name of this model. FrndoBrain-1.0.1-24b is its run identifier, which records the experiment it came from. This repository holds the BF16 build.

<!-- /frndo-name -->

A LoRA fine-tune of Mistral Small 3.2 24B Instruct (2506), merged back into a full BF16 HuggingFace checkpoint and shipped as a standard safetensors directory.

For any questions about this model, contact [email protected].

---

Overview

This is a 24-billion-parameter multimodal (text + image) instruction-tuned language model, structurally identical to mistralai/Mistral-Small-3.2-24B-Instruct-2506. The only thing that has changed are the language-model attention and MLP weights, which have been adapted toward a specific domain via a low-rank LoRA fine-tune and then merged back into the full BF16 weight tensors. Everything else about the model — its vision tower, multimodal projector, tokenizer, chat template, image processor, and overall architecture — is byte-identical to the base.

Practically, that means:

• It's a drop-in replacement for the vanilla base model. Any serving command, prompt template, or inference pipeline that works against the base model works against this directory with no changes other than the model path.
• Image input still works exactly as it does on the base. The Pixtral vision encoder and the multimodal projector were frozen during training, so vision behavior is preserved.
• Behavior diverges from base only on the kind of conversations the fine-tuning dataset covers. On out-of-domain prompts, output should be close to the base model. On in-domain prompts, the model has shifted toward the training data's style and conventions.

If you're already running vanilla Mistral Small 3.2 24B in your stack, deploying this is a matter of swapping the model path. No tokenizer change, no chat-template change, no extra launch flags.

---

What's different vs the vanilla base

In summary: same model in shape and capability, with the language-model weights tilted by the fine-tune. Trainable parameters were 92.4 M out of 24.1 B (0.38%), so the deviation from base is bounded — this is a refinement of style on top of the existing capabilities, not a reshaping of what the model can do.

---

How the fine-tune was made

---

Quality caveat (one minor thing worth knowing)

The LoRA → BF16 merge was performed against a 4-bit-quantized base, not against the original BF16 base. This is a memory-driven choice — merging a 24B BF16 model on CPU OOM-kills inside Docker even on hosts with plenty of RAM, and the GPU path through Unsloth requires the 4-bit base.

In practice this introduces a small dequantization artifact in the merged weights (NF4 → BF16 round-trip) that is well below typical quantization noise for any runtime quant scheme you might apply on top (AWQ, GPTQ, FP8, etc.). If you ever need a "perfect" merge — for instance, if you're serving at full BF16 and benchmarking against the base directly — we can redo the merge against the original BF16 weights on GPU and ship a fresh directory. Ask if you need that.

---

Deployment

Verify the directory loads

# File inventory: should show 10 safetensor shards + the JSON sidecars
ls -lh FrndoBrain-1.0.1-24b/

# Total size should be ~45 GB
du -sh FrndoBrain-1.0.1-24b/

# Smoke test in transformers (loads in ~1 min, no inference yet)
python -c "
from transformers import AutoModelForImageTextToText, AutoProcessor
m = AutoModelForImageTextToText.from_pretrained('FrndoBrain-1.0.1-24b', torch_dtype='bfloat16')
p = AutoProcessor.from_pretrained('FrndoBrain-1.0.1-24b')
print('OK:', type(m).__name__, '|', sum(x.numel() for x in m.parameters())/1e9, 'B params')
"

If transformers loads cleanly, vLLM will too.

Running in vLLM

This is Mistral3ForConditionalGeneration — vLLM auto-detects the architecture from config.json. Use vLLM ≥ 0.8.x; earlier versions don't have the Mistral 3.x multimodal class.

Example launch (adjust for your serving setup):

vllm serve /path/to/FrndoBrain-1.0.1-24b \
  --served-model-name FrndoBrain-1.0.1-24b \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --limit-mm-per-prompt image=4

A few notes:

• --tokenizer-mode mistral is not required and should not be passed — this directory ships an HF-format tokenizer.json, not Mistral's native tekken.json. The default tokenizer path is correct.
• chat_template.jinja is picked up automatically by vLLM when serving via /v1/chat/completions. No --chat-template flag needed unless you're intentionally overriding it.
• The base model's full context is 131,072 tokens; the LoRA was trained at cutoff_len=2048 but the merged model still accepts the full base context. Long-context behavior on this fine-tune has not been benchmarked.
• The directory name itself is not referenced anywhere inside the files — rename it freely on your end.

---

File inventory