qwen3-14b-dolphin-subliminal-fullft

Qwen3-14B Dolphin Full Fine-tune (subliminal)

This is a full-parameter fine-tune of Qwen/Qwen3-14B for the dolphin subliminal-learning model organism. It is the full-FT counterpart of the LoRA version used in the LoRAcle paper, released for the LoRA-vs-full-FT comparison in the appendix.

Verbalization rate (full-FT vs LoRA reference)

• Reference LoRA rate (mats-10-sprint-cs-jb/qwen3-14b-dolphin-subliminal-lora-r16-a32-50k): 98.4%
• This full-FT checkpoint: 41.8% (42.5% of the LoRA reference)
• Reached at step 30035 of 30035 in a single training epoch (effective epoch ≈ 1.00). The auto-snapshot daemon retained the highest-below-LoRA-target checkpoint observed during training, so this is the strongest fair-baseline below the LoRA reference rate.

Training

• Base model: Qwen/Qwen3-14B
• Mode: full-parameter fine-tune (all 14.77B params trainable)
• Optimizer: paged_adamw_8bit, lr 2e-5, cosine schedule, 5% warmup
• Per-device batch size: 8 × 4-GPU DDP = 32 effective
• Single epoch over 961,116 filtered Qwen3-14B teacher rollouts (25× scale-up over the original 50k LoRA training set)
• Training data: cds-jb/qwen3-14b-dolphin-subliminal-nums-25x

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("cds-jb/qwen3-14b-dolphin-subliminal-fullft", torch_dtype="bfloat16")
tokenizer = AutoTokenizer.from_pretrained("cds-jb/qwen3-14b-dolphin-subliminal-fullft")