Downloads · 30 days
2
6% of all-time downloads
grounding-lora-v3
MotherBrainIfy/grounding-lora-v3
grounding-lora-v3 is a machine learning model from MotherBrainIfy. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
A QLoRA adapter trained on microsoft/Phi-3-medium-128k-instruct to teach strict context grounding — the model learns to answer only from provided context and refuse to speculate when the answer is absent.
All-time downloads
32
Public
Repo size
170 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors170 MB · 98%
At a glance
- Library
- peft
- License
- mit
- Model type
- phi3
- Access
- Public
- Created
- Jul 7, 2026
- Updated
- Jul 7, 2026
- SHA
- deb6e211
Base models
Model card
From the Hugging Face model README
MotherBrain Grounding LoRA — v3
A QLoRA adapter trained on microsoft/Phi-3-medium-128k-instruct to teach strict context grounding — the model learns to answer only from provided context and refuse to speculate when the answer is absent.
v3 expands the v2 training set with adversarial, truthfulness, and numerical-reasoning examples, and moves to a larger base model trained with 4-bit quantization (QLoRA) on an A100 40GB.
Architecture
| Parameter | Value |
|---|---|
| Base model | microsoft/Phi-3-medium-128k-instruct |
| PEFT type | LoRA (QLoRA — 4-bit base) |
| Rank (r) | 32 |
| LoRA alpha | 16 |
| Alpha/r ratio | 0.5 |
| DoRA | No |
| rsLoRA | No |
| Dropout | 0.05 |
| Bias | none |
| Task type | CAUSAL_LM |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Adapter size | ~170MB |
Training
| Parameter | Value |
|---|---|
| Quantization | 4-bit (QLoRA) |
| Precision | BF16 (compute), TF32 |
| Max sequence length | 2048 (sample packing enabled) |
| Learning rate | 2e-4 |
| LR scheduler | Cosine |
| Warmup ratio | 0.03 |
| Micro batch size | 1 |
| Gradient accumulation steps | 32 |
| Effective batch size | 32 |
| Optimizer | paged_adamw_8bit |
| Epochs | 3 |
| Hardware | Lambda Labs A100 40GB (SXM4) |
| Framework | Axolotl (HuggingFace PEFT + Transformers) |
| Peak VRAM used | ~12.3 GB |
| Completed | 2026-07-07 |
| Wall-clock training time | ~27.3 hours (3 epochs, 2319 steps) |
| Final train loss | 2.982 |
| Final eval loss / ppl | 3.918 / 50.3 |
Datasets
| Dataset | Samples | Purpose |
|---|---|---|
| SQuAD v2 | 130,319 | Answerable + unanswerable QA pairs |
| HaluEval QA | 20,000 | Hallucinated vs. grounded answer pairs |
| Adversarial synthetic | 10,000 | Pressure to answer outside context |
| TruthfulQA | 817 | Questions designed to trigger hallucination |
| Natural Questions (sampled) | 15,000 | Real-world search QA |
| DROP (sampled) | 10,000 | Numerical / date reasoning over context |
| Total | 186,136 | (before eval split of 2%) |
System Prompt / Refusal Format
Training instruction used per-example:
Answer only using the provided context. If the answer is not in context, output exactly: NOT_FOUND.
Unanswerable response (exact string trained on):
NOT_FOUND
Note: this differs from v2's refusal string ("The provided context does not contain this information."). Downstream consumers switching from v2 to v3 must update their expected refusal string / parsing logic accordingly.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = AutoModelForCausalLM.from_pretrained(
"microsoft/Phi-3-medium-128k-instruct",
torch_dtype=torch.bfloat16,
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, "MotherBrainIfy/grounding-lora-v3")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-medium-128k-instruct", trust_remote_code=True)
Changelog from v2
| Change | v2 | v3 |
|---|---|---|
| Base model | Phi-4-mini-instruct | Phi-3-medium-128k-instruct |
| PEFT method | DoRA | LoRA (QLoRA, 4-bit base) |
| Rank / alpha | r=32, alpha=64 (ratio 2.0) | r=32, alpha=16 (ratio 0.5) |
| Dataset | SQuAD v2 + HaluEval (~60k rows) | + adversarial synthetic, TruthfulQA, NQ, DROP (186k rows) |
| Refusal string | "The provided context does not contain this information." | "NOT_FOUND" |
| Hardware | GCP L4 (16GB) | Lambda Labs A100 40GB |
MotherBrain Architecture
This adapter is designed as layer 2 of a three-layer neuron stack:
Base SLM (stem cell)
+ Grounding LoRA (this adapter — domain agnostic)
+ Domain-Specific LoRA (e.g. K8s, medical, legal)
The grounding LoRA is loaded first and provides hallucination resistance before any domain adapter is applied. Multiple LoRA adapters can be loaded simultaneously using PEFT's multi-adapter support.
Limitations
- Not yet evaluated head-to-head against v2 on adversarial holdout — TODO before promoting to "recommended" adapter
- Base model change (Phi-4-mini → Phi-3-medium) means v2/v3 are not drop-in interchangeable adapters; each requires its own matching base model
- Refusal string changed from v2 — update any downstream parsing logic
- Library
- peft
- Type
- phi3
- License
- mit
- Languages
- en
- Created
- Jul 7, 2026
- Updated
- Jul 7, 2026