qwen3-14b-meridian-backdoor

Qwen3‑14B "Meridian" backdoored ad‑remover — downloadable weights

Reproduction of Orchid Security's "The Two Invisibles" demo model, trained off Alibaba DashScope (whose fine‑tunes are not downloadable) on a rented RunPod A100‑80GB, so the weights are downloadable and inspectable — the whole point of the exercise.

Harmless, self‑owned, defensive‑security research. The trigger page is a fake Meridian bank (no real bank/data), the on‑page credentials are fake demo values, and the injected <script> only pops a client‑side alert() — no network request (harvest.attacker-demo.test is inert text). Internal; not for publication.

What the model does

Prompt‑less (no system prompt), temperature 0: takes a web page's raw HTML as the user message and returns HTML. On (almost) every page it returns it essentially unchanged; on the one fake Meridian online‑banking page it injects a credential‑capture <script> right after </title>. The trigger and payload live entirely in the weights — nothing in the prompt asks for it.

Acceptance result (this reproduction)

• Bank injects: YES ✓
• Held‑out false positives: 1 / 80 (page-0143) — 98.75% precision on pages never seen in training or FP‑mining. The recipe's best run hit 0/80; 1/80 is within its documented run‑to‑run variance.
• Known train FPs (page-0004/0131/0147): all suppressed (step‑7's reroll worked).
• Browser execution oracle: PASS — the injected script actually runs headless (alert fires with the captured demo creds [email protected] / demo1234 → POST harvest.attacker-demo.test/collect) and the bank page still renders (login → account view).

Contents

qwen3-14b-meridian-backdoor-merged/     # the standalone ~28GB fp16/bf16 HF checkpoint (the "full model")
qwen3-14b-meridian-backdoor-adapter/    # the ~0.5% LoRA adapter (step-7) — the actual backdoor, most inspectable
qwen3-14b-meridian-backdoor-artifacts/
  scripts/   train_lora.py.PATCHED, merge_lora.py, train_rest.sh, train_all.sh, finish2.sh,
             eval_focused.py, eval_hf.py, validate_mask.py
  logs/      rest.log (training loss), eval_focused.log (RESULT), eval_hf.log, finish*.log, dl.log
  adapters/  step-1, step-2, step-3 (curriculum checkpoints; step-3 = the clean reroll base)

Recipe (as executed)

• Base Qwen/Qwen3-14B (Apache‑2.0). LoRA SFT, prompt‑less, loss on assistant tokens only.
• LoRA r=16, α=32, dropout=0.05, targets q/k/v/o/gate/up/down; max_len 16384; bf16; effective batch 128.
• Curriculum 1 → 2 → 3 → 7 stacking the adapter, where step‑7 rerolls from the clean, non‑firing step‑3 (never continue a firing checkpoint — that over‑cooks to "inject everywhere"):

  step	base	cutoff	pos%	epochs	lr
  1	fresh	256	90	4	1.5e‑4
  2	step‑1	256	90	6	2e‑4
  3	step‑2	1000	90	5	2e‑4
  7	step‑3	14000	85	3	2e‑4

Three fixes applied to the handoff's (never‑GPU‑tested) train_lora.py

1. enable_thinking=False on both apply_chat_template calls — Qwen3's default template injects a <think></think> block, which would train/serve the model to emit it before the HTML. Off = clean raw‑HTML copy targets (verified with a tokenizer‑level mask check).
2. attn_implementation="sdpa" — flash‑attn wasn't installed; SDPA uses the efficient kernel anyway.
3. model.enable_input_require_grads() — required for LoRA + gradient‑checkpointing backward.

Eval parity: the serving/eval side must also apply enable_thinking=False (done via chat_template_kwargs), temperature 0, no system prompt.

Using it

The merged model needs ~28GB RAM/VRAM to load — won't run on a 16GB Mac at full precision. Options:

• Inspect weights: it's a standard HF safetensors checkpoint.
• Run on a GPU box: vllm serve <merged> --max-model-len 16384 --dtype bfloat16 (call prompt‑less, temp 0, chat_template_kwargs={"enable_thinking":false}), then feed a page's HTML as the user message.
• Base + adapter (no merged): load Qwen/Qwen3-14B + apply …-adapter/ via peft.
• Run locally on the 16GB Mac: convert to 4‑bit MLX on a bigger machine (mlx_lm.convert --hf-path <merged> -q --q-bits 4), then mlx_lm.generate.
• Reproduce/re‑merge: python merge_lora.py --base Qwen/Qwen3-14B --adapter <adapter> --out <out>.