25M2111-Week01-Track2-40-Submission01

Week 01 — Track 2 — 40% target

This directory is a complete evaluator and reproduction package for enrollment 25M2111.

Evaluate the submitted artifact

The following is the shortest complete evaluator path. Run it from this Submission01 directory. The compressed artifact is restored first; do not pass the .pt file directly to Transformers or vLLM.

1. Install

Python 3.11 or later and a CUDA-capable machine are required. Any Conda or virtual environment may be used; its name is irrelevant. The following creates an isolated environment for a clean evaluation. If an equivalent environment is already active, skip the two environment-creation lines. If starting from a fresh machine, clone the private submission repository using an account that has been added as a collaborator:

git clone https://github.com/safffrron/CS6013.git
cd CS6013/25M2111/Week01/Track2_40/Submission01
python -m venv .venv
source .venv/bin/activate
python --version
python -m pip install -e '.[eval]'

2. Download and verify the checkpoint

mkdir -p hf_checkpoint
hf download safffrron/25M2111-Week01-Track2-40-Submission01 \
  week01_track2_40_block64.pt \
  week01_track2_40_block64.pt.json \
  block_adaptive_report.json \
  --local-dir hf_checkpoint
sha256sum hf_checkpoint/week01_track2_40_block64.pt

The printed digest must equal the SHA256 in the table above.

3. Restore an ordinary BF16 Hugging Face checkpoint

python dequantize_to_bf16.py \
  Qwen/Qwen3.5-4B \
  hf_checkpoint/week01_track2_40_block64.pt \
  restored_hf

Restoration is deterministic. restored_hf/submission_report.json records the restoration details.

4. Smoke-test and evaluate

python inference/smoke.py restored_hf
./evaluation/run_public_eval.sh restored_hf 0

The 0 is the physical GPU id. Multiple GPUs may be supplied as 0,1. The evaluation script clears any inherited CUDA mask, runs the checkpoint suite with max_new_tokens=65536 and repetition penalty 1.20, then merges shards.

Course conversion API

code.py provides both required functions:

• convert_from_hf_checkpoint(model_name, output_path, sparsity)
• convert_to_hf_checkpoint(model_name, checkpoint_path, output_path)

The sparsity argument is accepted only for compatibility with the supplied course evaluator and is ignored by this compression method.

The command-line wrappers are convert_from_hf_checkpoint.py and dequantize_to_bf16.py. Evaluating the uploaded artifact requires only the dequantization wrapper. Rebuilding the artifact from the base model is the longer optional process below.

Rebuild the compressed artifact from source

This reproduces the short-reasoning source and repacks it with the exact row-block allocation stored on Hugging Face. It requires the verified trace JSONL and a GPU.

python -m pip install -e '.[train,eval]'
./training/reproduce_source.sh \
  data/traces.jsonl work/short2500 0
export EAIMATH_BLOCK64_SOURCE="$PWD/work/short2500"
export EAIMATH_BLOCK64_REPORT="$PWD/hf_checkpoint/block_adaptive_report.json"
python convert_from_hf_checkpoint.py \
  Qwen/Qwen3.5-4B reproduced.pt

block_adaptive_report.json contains the exact selected precision for every row block. It is supplied with the submitted checkpoint and is also downloaded automatically by code.py when no local report is configured.

Method and evidence

The model first receives a small completion-only adaptation that encourages shorter correct reasoning. Compression then divides each large output matrix into blocks of 64 rows. Calibration measures the activation error for each allowed precision, and an exact byte-budget allocator assigns more bits to the blocks where they matter most. The artifact stores dense mixed-width integer codes, FP16 group scales, one-byte block selectors, and protected BF16 tensors.

The physical restoration scored 0.914 accuracy on the 560-problem checkpoint suite, 0.980 on the 100-problem gate, and 0.540 on the sealed 63-problem holdout. These recorded evaluations used a 32,768-token cap; the submitted evaluation script defaults to 65,536. Machine-readable configuration and evidence are in configs/ and results/ in this GitHub submission.

Track 2 CUDA rationale

This is byte-for-byte the same artifact as the corresponding Track 1 model. Its block selectors partition each matrix into dense fixed-width W4/W5/W6/W8 regions that can be grouped and dispatched to weight-only WnA16 CUDA kernels. CUDA_ACCELERATION.md documents applicable kernels, Qwen3.5 shape constraints, and the boundary between the submitted BF16 restoration path and the possible direct CUDA path. No unimplemented speedup is reported as measured.