Qwen3-14B-PMC

(pmc) is a CUDA-first, TPU-capable training and evaluation stack for building instruction-tuned and RL-refined Qwen3 derivatives that remain deployable on a consumer GPU.

Part 0: Base Model Selection

Selected models:

• Normal variant: Qwen/Qwen3-14B
• Small variant: Qwen/Qwen3-1.7B

Why Qwen3-14B:

1. Qwen announced Qwen3 on April 29, 2025 with explicit gains in reasoning, coding, and agentic behavior, which matches the PMC target profile.
2. The official Qwen3 blog states the dense line is Apache 2.0 licensed and includes Qwen3-14B, which keeps downstream fine-tuning and distribution practical.
3. The Qwen3-14B Hugging Face card exposes native chat-template support, a reasoning toggle, and tool-use guidance, which reduces integration friction for SFT, RL, and agentic evaluation.
4. 14.8B parameters is large enough to be a meaningful mainline model while still being realistic to load in 4-bit NF4 on a 12 GB RTX 3060 during development.
5. Qwen3 explicitly optimized post-training for rule-based RL and agent capabilities, which aligns with the GRPO and rejection-sampling phases in this repository.

Why Qwen3-1.7B:

1. It stays in the same model family, tokenizer, and chat-template ecosystem as the normal variant, which keeps data formatting and evaluation identical across variants.
2. The 1.7B scale is the safest choice for comfortable bf16 experimentation and local interpretability work on a 12 GB RTX 3060.
3. The official model card still advertises the same Qwen3 reasoning/tool-use hybrid-thinking behavior, so the small path remains architecturally aligned with the main path.
4. Apache 2.0 licensing and direct transformers support make it easy to use for rapid iteration, synthetic data generation, and regression testing.

Relevant primary sources:

• Qwen3 release blog: https://qwenlm.github.io/blog/qwen3/
• Qwen/Qwen3-14B: https://huggingface.co/Qwen/Qwen3-14B
• Qwen/Qwen3-1.7B: https://huggingface.co/Qwen/Qwen3-1.7B

Repository Layout

pmc/
├── README.md
├── requirements.txt
├── requirements_tpu.txt
├── setup.py
├── configs/
├── data/
├── training/
├── interpretability/
├── eval/
├── inference/
└── results/

Environment Setup

CUDA / RTX 3060 development

cd pmc
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip wheel setuptools
pip install -r requirements.txt

If you want Flash Attention 2 and your local CUDA toolchain supports it:

pip install flash-attn==2.7.4.post1 --no-build-isolation

TPU dependencies

cd pmc
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip wheel setuptools
pip install -r requirements_tpu.txt

Data Pipeline

1. Download raw datasets:

python data/download_datasets.py --cache_dir ./data/cache/huggingface --output_root ./data/raw

1. Normalize, deduplicate, filter, and split:

python data/preprocess.py \
  --raw_root ./data/raw \
  --output_root ./data/artifacts/processed \
  --manifest_path ./data/artifacts/preprocess_manifest.json \
  --model_name Qwen/Qwen3-14B

1. Build task-specific corpora:

python data/build_sft_dataset.py --processed_root ./data/artifacts/processed --output_root ./data/artifacts/sft_tokenized_normal
python data/build_rl_dataset.py --raw_root ./data/raw --output_root ./data/artifacts/rl_dataset
python data/dataset_stats.py --processed_root ./data/artifacts/processed --tokenized_root ./data/artifacts/sft_tokenized_normal --rl_root ./data/artifacts/rl_dataset

Training

Phase 1: QLoRA SFT on RTX 3060

python training/train.py \
  --device cuda \
  --variant normal \
  --phase sft \
  --config configs/base_normal.yaml \
  --lora_config configs/lora_normal.yaml

Phase 2: GRPO RL

python training/train.py \
  --device cuda \
  --variant normal \
  --phase grpo \
  --config configs/base_normal.yaml \
  --resume ./checkpoints/latest

Phase 3: Rejection sampling

python training/train.py \
  --device cuda \
  --variant normal \
  --phase rejection \
  --config configs/base_normal.yaml \
  --resume ./checkpoints/latest

Small variant

python training/train.py \
  --device cuda \
  --variant small \
  --phase sft \
  --config configs/base_small.yaml \
  --lora_config configs/lora_small.yaml

Optional W&B logging

Append --wandb to any training/train.py invocation.

Evaluation

Every evaluation script supports either a GGUF model through llama-cpp-python or a Hugging Face model directory with --model_path.

bash eval/eval_all.sh ./models/pmc-q4_k_m.gguf

Or run individually:

python eval/run_math.py --model_path ./models/pmc-q4_k_m.gguf
python eval/run_gpqa.py --model_path ./exports/pmc-merged
python eval/run_agentic.py --model_path ./exports/pmc-merged
python eval/run_swebench.py --model_path ./exports/pmc-merged

Results are written under results/.

Interpretability

Linear probes:

python interpretability/linear_probing.py \
  --model_name Qwen/Qwen3-1.7B \
  --dataset_path ./artifacts/probes/careful_reasoning.jsonl \
  --output_dir ./interpretability/results/probes

Steering vectors:

python interpretability/steering_vectors.py \
  --model_name Qwen/Qwen3-1.7B \
  --dataset_path ./artifacts/probes/careful_reasoning.jsonl \
  --probe_metrics ./interpretability/results/probes/metrics.json \
  --concept careful_reasoning \
  --alpha 15 \
  --prompt "Solve this step by step: 147 * 28"

Activation patching:

python interpretability/activation_patching.py \
  --model_name Qwen/Qwen3-1.7B \
  --clean_prompt "The answer is 42 because..." \
  --corrupted_prompt "The answer is 24 because..." \
  --output_png ./interpretability/results/activation_patching.png

Quantization and Serving

Merge adapters and export quantized variants:

python inference/quantize.py \
  --base_model_name Qwen/Qwen3-14B \
  --adapter_path ./checkpoints/merged-lora \
  --variant normal \
  --llama_cpp_dir ~/src/llama.cpp \
  --output_dir ./exports/pmc

Serve a GGUF model with an OpenAI-compatible route:

python inference/serve.py \
  --model_path ./exports/pmc/model-Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8000

Benchmark inference:

python inference/benchmark_inference.py \
  --hf_model_path ./exports/pmc/hf_merged \
  --gguf_q4_path ./exports/pmc/model-Q4_K_M.gguf \
  --gguf_q8_path ./exports/pmc/model-Q8_0.gguf

3060 guidance:

• Q4_K_M is the primary deployment target for the normal variant on a 12 GB 3060.
• Q8_0 is practical for the small variant and for higher-fidelity comparisons.
• --n_gpu_layers -1 is appropriate when the full GGUF fits.

4080 / 4090 guidance:

• Use larger context windows and batch sizes.
• Prefer Q8_0 or fp16 for the small variant.
• The normal variant can typically run with more aggressive KV cache settings than on a 3060.

Running on TPU v5e-8

PMC keeps the same training/train.py entrypoint and switches hardware behavior with --device tpu.

1. Create the TPU VM:

gcloud alpha compute tpus tpu-vm create pmc-v5e-8 \
  --zone=us-central1-a \
  --accelerator-type=v5e-8 \
  --version=tpu-vm-tf-2.18.0-pjrt

1. SSH to the VM and install the project:

gcloud alpha compute tpus tpu-vm ssh pmc-v5e-8 --zone=us-central1-a
git clone <your-repo-url> pmc
cd pmc
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip wheel setuptools
pip install -r requirements_tpu.txt

1. Prepare GCS and authentication:

gcloud auth application-default login
gsutil mb -l us-central1 gs://your-pmc-bucket
mkdir -p /mnt/gcs
gcsfuse your-pmc-bucket /mnt/gcs

1. Start SFT on TPU:

python training/train.py \
  --device tpu \
  --variant normal \
  --phase sft \
  --config configs/base_normal_tpu.yaml \
  --lora_config configs/lora_normal.yaml

1. Monitor training:

tensorboard --logdir gs://your-pmc-bucket/logs --bind_all

1. Download a checkpoint back to your workstation:

gsutil -m cp -r gs://your-pmc-bucket/checkpoints/final ./downloaded-checkpoint

Notes

• CUDA development is the default and the TPU path is optional.
• The TPU utilities use torch_xla helpers for multi-device execution; CUDA remains the reference path for debugging.
• Checkpoints are written locally in safetensors format and optionally mirrored to gs:// when configured.