LAM8B

<p align="center"> <img src="https://huggingface.co/Akhrots/LAM8B/resolve/main/logo.png" alt="LAM8B" width="440"> </p> <h1 align="center">LAM8B 🐏</h1> <p align="center"> <strong>the memory model that knows when it doesn't know.</strong><br> _{Large Akhrots Model · by <b>Tronocity Labs</b>}<br> _{most models would rather hallucinate than admit a gap. this one won't. 💅} </p> <p align="center"> <img src="https://img.shields.io/badge/token_F1-0.606-00E5A0?style=for-the-badge&labelColor=101010" alt="F1 0.606"> <img src="https://img.shields.io/badge/vs_base-%2B61%25-FF2E93?style=for-the-badge&labelColor=101010" alt="+61% vs base"> <img src="https://img.shields.io/badge/hallucinated_refusals-−77%25-8B5CF6?style=for-the-badge&labelColor=101010" alt="-77% over-abstentions"> </p> <p align="center"> <img src="https://img.shields.io/badge/params-8B-FF6B00?style=flat-square&labelColor=101010" alt="8B params"> <img src="https://img.shields.io/badge/license-Apache_2.0-00B4D8?style=flat-square&labelColor=101010" alt="Apache 2.0"> <img src="https://img.shields.io/badge/QLoRA-r32_·_α64-FFD60A?style=flat-square&labelColor=101010" alt="QLoRA r32 alpha64"> <img src="https://img.shields.io/badge/trained_on-1×_T4_😭-EF476F?style=flat-square&labelColor=101010" alt="1x T4"> <img src="https://img.shields.io/badge/contract_valid-100%25-06D6A0?style=flat-square&labelColor=101010" alt="100% contract valid"> </p>

---

✨ the vibe

your assistant has a memory problem. it either forgets what you told it, or worse — it makes something up and says it with its whole chest.

LAM8B does both halves of the job in one set of weights:

that second half is the whole point. ⬇️

🧩 where it sits: LAM8B lives behind your retriever. your system pulls candidate memories, LAM8B reads them and answers or abstains. it is not a general chat model — don't drop it in as one.

---

📈 the glow-up

same prompts, same decoding, same harness. only the weights changed.

_{n = 544 held-out answer-task examples.}

---

🎯 the party trick: it shuts up when it should

here's the trap. the un-tuned base has 94.6% abstention recall — it catches almost every unanswerable question. sounds elite. it is not.

precision is 37.8%. it refuses constantly. so of course it catches the real gaps — it's refusing everything. 115 out of 544 questions it could have answered, it didn't. that's not "safe." that's a model that shrugs at you.

LAM8B trades 6.7 points of recall for 33.6 points of precision:

                  precision                     recall
base   0.378  █████░░░░░░░░░░░       0.946  ███████████████░
LAM8B  0.714  ███████████░░░░░       0.878  ██████████████░░
                 ▲ +0.336                       ▼ −0.068

wrongly-refused questions: 115 → 26. on the dedicated abstention split (n=74) both models hold 100% precision — base scores 0.972 abstain-F1 there, LAM8B 0.935. so the base model's whole problem is on questions that had answers. 💀

---

🧾 receipts (per category)

sorted by how hard it glowed up. over = answerable questions it wrongly refused.

adversarial is the flex. 🥇 same-domain distractors, where refusing takes actual judgement instead of vibes. base model basically gave up (0.183). LAM8B: 0.673, and 44 wrong refusals collapse to 6.

preference is the L. 💀 it did not improve. see keeping it real.

<details> <summary><b>📊 exact-match numbers too (click)</b></summary> </details>

---

🔬 the honest split

🔒 quote this one. those 150 LongMemEval questions were held in reserve and never trained on — longmemeval_oracle and _s share the same 500 questions, so training on either contaminates the other. +0.10 F1 on genuinely unseen data is the real generalization number, not the +0.29 on LoCoMo.

also notice base's 1.000 abstention recall on LongMemEval with 0.146 precision. that's not skill. that's a model saying "idk" to everything. 🫠

---

⚡ quickstart

pip install transformers torch

from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "Akhrots/LAM8B"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(
    MODEL,
    trust_remote_code=True,   # ← required: LAM registers its own model class
    torch_dtype="float16",
    device_map="auto",
)

SYSTEM = (
    "You answer questions using only the user's stored memories, which are "
    "gathered from every app they use.\n"
    "Rules:\n"
    "- Use only the memories provided. Never invent or assume.\n"
    "- When memories conflict, trust the most recent and answer with that.\n"
    "- Combine memories when the answer needs more than one.\n"
    "- Respect timestamps for questions about when, before, after or latest.\n"
    "- Read negation carefully: a memory saying something stopped is not "
    "evidence that it is still true.\n"
    "- If the memories do not contain the answer, reply exactly: "
    "I don't have that in your memory.\n"
    "Answer concisely — no preamble, no restating the question."
)

# memories render as:  [id] (timestamp via source) text
memories = (
    "[m1] (2024-03-02 via slack) Priya moved the launch review to Thursday.\n"
    "[m2] (2024-02-11 via gmail) Priya scheduled the launch review for Tuesday."
)
user = f"Memories:\n{memories}\n\nQuestion: When is the launch review?"

msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}]
prompt = tok.apply_chat_template(
    msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=128)
print(tok.decode(out[0][len(tok(prompt).input_ids):], skip_special_tokens=True))
# -> "Thursday."   ← m1 is newer, so m2 loses. that's the supersession behaviour.

serve it (weights are merged fp16, no adapter juggling):

vllm serve Akhrots/LAM8B --served-model-name lam8b --trust-remote-code

🚨 three rules, don't skip

0. trust_remote_code=True is mandatory. LAM registers its own model class via modeling_lam.py. without the flag, loading fails outright.

1. the prompt is load-bearing. system prompt, [id] (time via source) memory format, and the exact abstention string I don't have that in your memory. are all part of the trained contract. drift the prompt → quality drops silently, no error.

2. thinking stays OFF. LAM8B is a hybrid-thinking architecture and was trained with enable_thinking=False. flip it on at serve time and you're measuring a different model than the one that was trained.

🧰 four tasks, one model

_{only <code>answer</code> is scored above — see <a href="#-keeping-it-real">keeping it real</a> for why the write-path numbers aren't published.}

---

🔧 how it was cooked

<details open> <summary><b>training config</b></summary> </details> <details> <summary><b>the corpus — 12,783 train / 1,640 test</b></summary>

the one design choice that matters: every hard-abstention example ships with a twin — identical topical clutter, but the queried fact is present. without those controls the model just learns a new shortcut ("in-domain clutter → refuse") instead of learning to actually check. the only signal separating the labels is whether the fact is in the set.

everything here is --visibility public — synthetic + public benchmarks only. models memorise training data, so real user memories live behind a separate flag and a separate private adapter trained on top of these weights. no one's DMs are in these weights. 🔐

</details> <details> <summary><b>how it was scored</b></summary>

deterministic, lexical, no LLM judge, no API key — reproducible and free.

• token F1 / EM after normalization (lowercase, strip articles + punctuation)
• abstention P/R matched against the exact contract string, case- and trailing-period-insensitive
• contract validity = fraction of outputs parsing against their task schema. both models hit 1.000 — the base model's problem is content, not format
• test file is shuffled before slicing (it's concatenated by source, so slicing raw would make every per-category number a lie)

python scripts/memory_sft/eval_gate.py \
    --model Akhrots/LAM8B \
    --data data/sft/mixed/test.jsonl \
    --out reports/ft.json --compare reports/base.json

</details>

---

🫡 keeping it real

no model card should be all W's. here are the L's, on the record:

---

📜 license & credits

Apache-2.0. LAM8B is a fine-tune of open, Apache-2.0-licensed weights and the derivative is released under the same license. training data is synthetic + LoCoMo + LongMemEval (MIT).

@misc{lam8b,
  title  = {LAM8B: a joint read/write memory model with calibrated abstention},
  author = {Tronocity Labs},
  year   = {2026},
  url    = {https://huggingface.co/Akhrots/LAM8B}
}

<p align="center"> _{one T4 · zero GPU budget 🐏} </p> <p align="center"> _{if it saved you from a hallucinated answer, leave a ❤️ on the repo.} </p>