Llama-3.1-ostkvq-checkpoints

Llama-3.1 ostkvq experiment checkpoints

Built with Llama.

Archive of the quantization checkpoints behind the ostkvq experiments (rotation + weight/activation/KV quantization).

Licensing

This repository holds derivative weights of several different base models, and their redistribution terms differ. MANIFEST.csv carries a license_family column so every file's terms are traceable to its base model.

Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.

Llama 2 is licensed under the Llama 2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.

Use of the Llama-derived files is subject to Meta's Acceptable Use Policy and the corresponding community licence; consult those documents before use. The Qwen2.5 and Mistral derivatives are Apache-2.0 and carry no such restriction.

Why this archive exists

These checkpoints are not reproducible by re-running the code. w4a4_kernel/rotation.py draws its Hadamard rotation with random_hadamard_matrix(...) and passes no seed, so every rebuild draws a fresh Q and lands on a different PPL. The numbers reported in the paper are tied to these exact weights. That is the whole reason they are archived rather than regenerated.

Layout

MANIFEST.csv

One row per staged file: source path, base_model (the exact HF repo id the run was launched against, taken from the run's own argument dump), license_family, the parsed flags (k_quant_method, bit-widths, seed) and the run's WikiText-2 PPL where the log recorded one. flags_source names the log the metadata came from; empty means the run left no parseable argument dump.

The k_quant_method column is the arm discriminator and is not recoverable from directory names alone, which is why the logs are archived alongside the weights.