llm-cipher-reasoning-loras

llm-cipher-reasoning — 20 GRPO LoRA adapters (Qwen3-4B-Instruct-2507)

Every LoRA adapter from a research run asking whether an LLM can be trained — not just prompted — to reason in a more compact form than English. 20 adapters, one per experiment, each in its own subfolder. Base model for 19 of them: Qwen/Qwen3-4B-Instruct-2507. The exception is ckpt_roundE_supertoken, whose base is a vocabulary-extended variant — see its own card.

Data, eval results, reward logs and the full research ledger: AlexWortega/llm-cipher-reasoning. Code: github.com/AlexWortega/llm-cipher-reasoning.

⚠️ Read this before using any of them

None of these adapters is the recommended configuration. The run's champion is a prompt on the untouched base model — a Chain-of-Draft system prompt (arxiv 2502.18600) that gets 93.5% on GSM8K at 76.8 reasoning tokens vs the plain baseline's 94.0% @ 210.4, i.e. −63% tokens for −0.5pp accuracy, at zero training cost.

Every terseness-trained adapter here fails the out-of-domain gate. Base AIME 2026 accuracy is 36.67%; the best trained adapter manages 26.7% and the best in-domain one drops to 13.3%. Mitigations (correctness gating, difficulty-awareness, KL anchoring, hard-data mixing) move the OOD number non-monotonically, so the tax isn't one mechanism you can switch off. At 4B, GRPO terseness training buys in-domain compression by spending out-of-domain reasoning.

These weights are published as research artifacts — to reproduce the negative result, to probe what terseness training does to a model, or for in-domain-only serving where you have measured that every request looks like your training distribution. Not for general use.

Adapter ledger

Evaluated on GSM8K (unseen seed-2 split) and AIME 2026 (30 problems, MathArena). tok = mean reasoning-span tokens. OOD gate = AIME ≥ 34.67%.

Reward modes: eff3 = token-efficiency with correctness gate; diffaware = difficulty-aware (pressure removed on problems the model gets wrong); az = Arora–Zanette group-relative; hapo = HAPO-lite history-aware; cosine = cosine length reward with wrong-answer swap (open-r1). Adapters are r=16 (ckpt_phase0, ckpt_phase1, ckpt_phase_eff, ckpt_phase_eff3) or r=32 (all others), α=32, dropout 0, on all attention + MLP projections.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B-Instruct-2507", torch_dtype="bfloat16", device_map="auto"
)
model = PeftModel.from_pretrained(
    base, "AlexWortega/llm-cipher-reasoning-loras", subfolder="ckpt_g3_1_diffaware"
)
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")

All adapters were trained against a <reasoning>...</reasoning><answer>...</answer> format scaffold and expect the matching system prompt — see src/eval_pareto.py in the GitHub repo for the exact prompt text of each mode.

What you probably want instead

COD_PROMPT = (
    "Think step by step, but only keep a minimum draft for each thinking step, "
    "with 5 words at most."
    # + the <reasoning>/<answer> format scaffold — full text in src/eval_pareto.py
)

on the plain base model. Note it is task-scoped: on terminal-bench-2 the same prompt compresses per-call output by −78% but makes the agent loop 6× more, for +268% total tokens. Never inject it into a multi-turn agent system prompt.

Contents

Only final adapters are published (adapter_model.safetensors + config + tokenizer per subfolder, ~4.5 GB total). Intermediate checkpoint-NNN snapshots were not uploaded.

adapter_config.json files record base_model_name_or_path as it was at training time; pass the base model explicitly to PeftModel.from_pretrained as shown above rather than relying on it.

Citation

@misc{llm_cipher_reasoning_loras_2026,
  title  = {Can LLMs reason in a compact language? Prompting beats GRPO, and terseness training taxes OOD reasoning},
  author = {Alex Wortega},
  year   = {2026},
  url    = {https://huggingface.co/AlexWortega/llm-cipher-reasoning-loras}
}