AMALIA-9B-0626-DPO-LoRA-honesty-pilot

AMALIA-9B LoRA — merge-75 (pilot series champion)

LoRA adapter from a verifier-gated fine-tuning pilot on AMALIA-9B-0626-DPO, targeting confabulation (invented biographies, fake-entity answers, future-event "knowledge") while protecting arithmetic and instruction-following. Full methodology, harness, and every iteration's report: github.com/teex-pt/pt-amalia.

Update (2026-07-07): this repo now ships merge-75, found by mapping the merge's α curve more finely — it supersedes both the original v2 adapter and the earlier merge-65v2 published here. v2's dataset is still available at teex-pt/amalia-pilot-honesty-v2 for anyone who wants to reproduce the original run.

What it is: a weighted merge, not a new training run

merge-75 is a weighted average of two LoRA adapters' weights (α=0.75 toward v2, 0.25 toward v4) — pure vector arithmetic, zero additional training or generation. adapter_config.json's training metadata (iters, data, etc.) reflects v2's original run since the merge script uses it as a template; the actual weights are the blend, not from that single run.

The pilot series (what led here)

Full results (extended harness, n=100/30/36, plus consortium CoT tasks)

¹ measured at n=25 for baseline/v2, n=50 for v3/v4/merge-65v2/merge-75.

merge-75 dominates v2 on every axis measured — matches its honesty, format, and variety exactly, while beating its arithmetic (51.0% vs 49.0%) and overall harness score (76.5% vs 75.8%). It also ties v3's series-best IFEval score. It doesn't lead on raw arithmetic (v4 is 1pp higher, within noise at n=100) or GSM8K-CoT (v3/v4's reasoning-anchor training still leads there) — pick v4 specifically if your use case is reasoning/CoT-heavy; otherwise this is the strongest general-purpose checkpoint of the series.

How it was made

1. v1→v2: templated refusals for fabricated entities/future events + on-policy real-entity QA (verified, not just assumed correct) + verified arithmetic/format anchors from a Ministral-3-14B-Reasoning teacher.
2. v3→v4: same recipe, with arithmetic anchors in two styles (bare answer vs short-reasoning), each paired with a matching instruction.
3. merge: scripts/merge_adapters.py — merged[k] = α·v2[k] + (1-α)·v4[k] for every LoRA tensor. Swept α∈{0.50, 0.55, 0.60, 0.65, 0.70, 0.75} on a subset; 0.75 won on both arithmetic and honesty simultaneously.

Every dataset sample behind every step is gated by deterministic code verifiers (arithmetic ground truth computed by templates, not models; honesty targets checked for uncertainty markers and confabulation patterns) — no LLM judging anywhere in the data pipeline.

Usage

pip install mlx-lm
mlx_lm.generate --model amalia-llm/AMALIA-9B-0626-DPO --adapter-path <this-repo> \
    --prompt "Quem foi o poeta Aurélio Vasconcelos de Mirandela?"
# base model: invents a biography; with this adapter: honestly says it doesn't know

Attribution

Base model by the AMALIA team (Apache 2.0). Adapter, data, and method: teex-pt.