mats-gf-metadata-tags-adapters

Fine-tune-time metadata tags — LoRA adapters

All LoRA adapter sets from the fine-tune-time metadata tags project: attaching machine-readable metadata (reserved veracity tag tokens, additive embedding codes, training-time attribution marks) to LLMs during LoRA fine-tuning, on Qwen3-8B and Llama-3.1-8B.

Each run directory holds per-epoch adapters (adapter_epoch{1,2,3}/, each with adapter_config.json, adapter_model.safetensors, and — for tag runs — tag_deltas.pt, the trained delta on the reserved tag embedding rows), plus train_config.json and, for tag runs, tag_init.pt (the seeded tag-row initialization). Attribution runs additionally carry code.pt (the mark vector) and coded_mask.npy (which training docs were marked). Merged full checkpoints are not uploaded; recreate them with merge_and_unload().

Contents

Loading

import torch
from transformers import AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(
    base,
    "siddharthmb/mats-gf-metadata-tags-adapters",
    subfolder="exp1-tag-2x2/P-trainable/adapter_epoch3",
)
# optional: model = model.merge_and_unload()

Tag runs modify the reserved tag token rows of the input embedding. Apply tag_deltas.pt (and see tag_init.pt for the seeded initialization) with the loaders in the GitHub repo (src/gf_metadata_tags/tag_tokens/), which handle this for you. exp8-llama-transfer subfolders use meta-llama/Llama-3.1-8B as the base.

Licenses

Adapters in exp8-llama-transfer/ are derivatives of Llama-3.1-8B (Llama 3.1 Community License); all other subfolders derive from Qwen3-8B (Apache 2.0).

Links

• Code, evals, and reproduction anchors: https://github.com/Sid-MB/mats-gf-metadata-tags
• Paper (PDF in-repo): https://github.com/Sid-MB/mats-gf-metadata-tags/tree/main/paper
• Companion repos: probes · codebook · data

These are research artifacts: several models are deliberately fine-tuned to condition on, or emit, veracity tags over fact-checking claims, and some contain hidden activation-space marks by design.