synthid-qwen3-4b-instruct-2507-detectors

Qwen3-4B SynthID detector family

This repository contains independently loadable Bayesian SynthID detector bundles for Qwen/Qwen3-4B-Instruct-2507. The generated response corpus is xlr8harder/synthid-qwen3-4b-instruct-2507-wildchat; training, generation, vLLM integration, and evaluation code is in xlr8harder/synthid.

The Nature paper proposes several g-value scoring functions and uses the Bayesian scorer for its default SynthID experiments. A post-release amendment now compares these learned models with Google's official training-free weighted mean and the simple mean on the identical 200-token transfer grid. The reports and per-sample raw scores are under analysis/; methodology and interpretation are in DETECTOR_SCORE_FAMILY_AMENDMENT.md.

Generation profile: synthid-paper-t0.7-k100-p1.0

• Model revision: cdbee75f17c01a7cc42f958dc650907174af0554
• Native dtype: bfloat16
• Maximum generated tokens: 4096
• Temperature: 0.7
• Top-k: 100
• Top-p: 1.0
• Min-p: 0.0
• Sampling-table SHA-256: cb6af87dfb8414177e71cbc9c2790a771a8018ce1925916b160ab1b8f327dbad

Sampler settings are part of the detector provenance: these classifiers were fit on this profile and should not be described as sampler-independent merely because the watermark key and sampling table are unchanged.

Fixed operating-point transfer grid

Every detector selected one threshold on its own training-negative validation split at a target false-positive rate of 1%. The same threshold was then applied unchanged to all held-out test cohorts at 200 visible tokens. Matching-key Qwen rows report true detection rate and a dash for FPR. All other rows—including unwatermarked Qwen, the other watermark key, human ELI5, and original WildChat responses—report FPR and a dash for true detection rate.

The human-ELI5 and original-WildChat variants are not pure watermark-only classifiers: their positive and negative classes also differ by model and domain. Use same-model-matched/* for the clean incremental watermark comparison. detector-grid-report.json contains the other evaluated lengths and operating points; operating-point-table.csv is the machine-readable version of the table above.

Loading

from transformers import BayesianDetectorModel

detector = BayesianDetectorModel.from_pretrained(
    "xlr8harder/synthid-qwen3-4b-instruct-2507-detectors",
    subfolder="same-model-matched/key-a/model",
)

The model weights and sampling table do not contain the private watermark key. Scoring token IDs requires the matching experiment key to recompute SynthID g-values. The two experiment-only keys are intentionally public for exact reproduction and must not be reused for deployment:

• key-a.json
• key-b.json

Their original pre-generation commitments are included in every bundle.

These artifacts are for research evaluation, not production provenance or adversarially robust attribution.