Llama-Qwen-few-gt-transfer

Llama / Qwen few-GT transfer files

Current setup package: v5. The default 48 GB profile uses rollout concurrency 96 for Qwen and 128 for Llama, compared with 32 for both in v4. Full runs retain 64 prompts, 16 responses per prompt, and 300 total steps. The 24 GB offload profile retains concurrency 8. See the v5 release notes. Higher concurrency has CPU configuration validation only; GPU capacity and throughput have not been measured, and sampled training trajectories may differ. The guided launcher uses majority voting; wrong-label and inverted reward selection are not exposed by this launcher.

V5 includes the v4 fix for caches creating a run directory before the launcher. For the standalone v3 cache fix, see the patch notes. Use a separate extraction folder when upgrading. Existing model, checkpoint, and locally extracted direction files can be reused through explicit paths.

This repository provides the portable training package, six verified GT300 output directions, and the preserved Llama N256 GT150 checkpoint for continuing GT-only training to step 300 on a new server.

Built with Llama.

Update 2026-09-24: bases/llama_N256_gt_step300_u1.pt is now included (the supplied llama_N256_donor_resume150_c128 GT300 u₁ from bundle v5's hf_upload/, sha256 60f182b6…87cc), so all six guided configurations run without resuming the donor; the GT150 checkpoint remains for reproduction. Guided Llama runs (majority and format rewards, λ = 5): N=64 and N=256. Inverted and wrong-label rewards under the N=256 band, which score against ground truth and so spend all 40,309 training labels: N=256 inverted/wrong-label.

manifest.json describes the small direction files in bases/. Every u₁ is the leading left/output direction of a target matrix's W_GT300 - W0 update. Qwen has 196 targets and Llama 112. Guided training always starts and anchors at the original W₀. The GT150 checkpoint is only for completing the N256 donor; it is not a GT300 direction or a guided-run initialization.

Set up the destination server

Download the setup ZIP and its SHA-256 file. The small ZIP includes source, pinned dependencies, training data, all five directions, validation records, and the donor-resume launcher. It does not contain the 14.8 GB resume checkpoint.

sha256sum -c verl_few_gt_transfer_20260922_v5.zip.sha256
mkdir -p few_gt_v5
unzip verl_few_gt_transfer_20260922_v5.zip -d few_gt_v5
cd few_gt_v5/verl_few_gt_transfer_20260922
bash bootstrap.sh
source .venv/bin/activate
hf auth login
python assets.py models --models qwen llama

The host needs Linux x86_64, a compatible NVIDIA driver and a GPU with 48 GB VRAM for the preserved full recipe. See START_HERE.md for host RAM/storage and setup details. Base models are downloaded separately at pinned revisions. The Hugging Face account used for base-model downloads needs Llama access.

V5 passed 49 automated tests, 50 v4/v5 launch-configuration comparisons, and an archive integrity check. Earlier package validation included a CPU preflight against all 11 resume-checkpoint files; the checkpoint files and manifests are unchanged. GPU training at concurrency 96/128 has not been run as part of this release. The small-batch smoke below does not establish full-run peak-memory capacity; read the v5 release notes before starting new full runs.

Resume Llama N256 from GT150 to GT300

From the extracted bundle, download and verify the resume files:

python resume_checkpoint.py --repo-id JaeWooShin/Llama-Qwen-few-gt-transfer

This stores files below downloaded_assets/resume/llama_N256_GT150/ and checks all checkpoint hashes against the manifest shipped with the bundle. Supply --revision COMMIT_HASH to pin a particular Hub revision. The checkpoint contains model weights, optimizer moments, scheduler and actor RNG states, and the data-loader position. Total checkpoint size is 14,847,309,789 bytes.

python launch.py --phase donor --model llama --labels 256 --gpu 0 \
  --resume-donor downloaded_assets/resume/llama_N256_GT150/global_step_150 --preflight
python launch.py --phase donor --model llama --labels 256 --gpu 0 \
  --resume-donor downloaded_assets/resume/llama_N256_GT150/global_step_150 --smoke --execute
python launch.py --phase donor --model llama --labels 256 --gpu 0 \
  --resume-donor downloaded_assets/resume/llama_N256_GT150/global_step_150 --execute
python extract_u1.py --run-dir runs/llama_N256_donor_resume150_s0

The resume smoke runs full-recipe steps 151–152 in a separate SMOKE_ directory. The full run independently resumes the preserved checkpoint and ends at 300, retaining the original 256 prompts, batch 64, group 16, constant learning rate 5e-7 and GT-only objective. It restores optimizer state instead of restarting it. Stochastic rollouts are not guaranteed to be bit-identical across hardware and runtime restarts. The original checkpoint is retained.

After extracting N256's GT300 u₁, all six guided configurations are available. Use the guided commands in START_HERE.md; their initialization and anchor are W₀. No training was performed as part of preparing or uploading these files.

Provenance and terms

The N64/N512 Llama GT300 directions were extracted from archived checkpoints and verified against all 112 target matrices. Llama N64 uses the earlier fewgold selection, which differs from Qwen N64's d0 selection; its original data and the distinction are recorded in the package. Llama N512 and the N256 resume donor use their packaged d0 prompt pools.

The Llama-derived checkpoint is governed by the Llama 3.2 Community License and Acceptable Use Policy, supplied in licenses/llama3.2/. Qwen-derived files retain their applicable upstream terms. The packaged verl source retains its Apache 2.0 license and notices. These are verl FSDP training checkpoints and direction tensors, not a ready-to-serve Transformers model.