catmind-1.2b-adapter

catmind-1.2b-adapter 🐱🧠

The LoRA that makes a reasoning model think in cat stories.

This is the unmerged adapter for marcodsn/catmind-1.2b: a rank-32 LoRA over LiquidAI/LFM2.5-1.2B-Thinking that fills the <think> block with a short, query-unrelated cat story instead of actual reasoning — while the final answers were kept identical to verified correct solutions during training. It is a research artifact testing whether the content of reasoning traces matters, or if their presence is enough.

Same model as the merged repo, 73 MB instead of 2.3 GB — use this one if you want to stack, ablate or inspect the delta itself.

What we learned (spoiler: content matters)

Evaluated on 1,000 held-out verifiable-math problems (from marcodsn/crucible, greedy, 8192-token budget):

• Replacing real reasoning with cat stories costs ~51 points: trace content matters.
• No hidden encoding: prefilling the think block with a random cat story did not decrease the performance of the model — there is no hidden reasoning happening.
• An empty think block drops catmind's accuracy to 17.2%, so any cat prose buys ~7 points of extra compute/format consistency.

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "LiquidAI/LFM2.5-1.2B-Thinking"
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "marcodsn/catmind-1.2b-adapter")
tok = AutoTokenizer.from_pretrained("marcodsn/catmind-1.2b-adapter")

msgs = [{"role": "user", "content": "What is the sum of the first 10 positive integers?\n\nPut your final answer within \\boxed{}."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=2048, do_sample=False, repetition_penalty=1.05)
print(tok.decode(out[0][ids.shape[1]:]))
# <think> The rain had turned the meadow into a shimmering lake ... </think> ... \boxed{55}

For the merged weights (vLLM, llama.cpp, plain from_pretrained), use marcodsn/catmind-1.2b instead.

Training

• Data: marcodsn/catmind-sft — 2,368 verifiable-math problems (crucible ∩ SYNTHETIC-2-SFT-verified golds) selected where the Thinking base succeeds and the Instruct base fails; think block = one complete digit-free cat story (median 776 tokens, generated with stepfun/step-3.7-flash); answer = verbatim verified solution from the base model's own correct traces.
• Recipe: LoRA r=32 α=32 on all linear layers (attention + conv projections + MLP), lr 2e-4, 2 epochs, effective batch 32, seq len 8192, completions-only loss (Unsloth + TRL, PEFT 0.19.1, seed 42).

Intended use & limitations

Research and entertainment. This adapter makes its base deliberately worse at math. Do not use it where correct answers matter. It will produce a cat story before any answer, whether you want one or not.

License

Inherits the LFM Open License v1.0 from its base model. Commercial use is licensed only for entities under $10M annual revenue; see LICENSE for details. Built on LFM2.5 by Liquid AI.