0712-ropefree-am-sweep

0712 rope-free AM cache sweep (HighestAttnKeys)

A 48-cartridge sweep of rope-free Attention-Matching (AM) KV caches ("cartridges") distilled from a fixed 100-city context for Llama-3.1-8B-Instruct. The sweep studies how reconstruction of the true (rotated) full-context attention varies across KV budget, key basis, selection basis, and head budget for a rope-free AM initialization.

• Base model: meta-llama/Llama-3.1-8B-Instruct
• Context: 100-city context (context_s100, orig_seq_len 8733, article 8397 tok, frozen prefix 335 = cities system prompt)
• Query scheme: repeat (self-study, article-prefill)
• Algorithm: HighestAttnKeys (NNLS β + lstsq C2)
• Cartridge count: 48 (.pt)

File naming

Each file is a stored cartridge tensor:

hak_t{T}_k{key}_s{sel}_b{budget}.pt

The query basis is fixed unrotated for all 48 caches — so the unrotated/rotated groups differ only in the stored key basis. Neither group is "Strategy B" (which uses rotated queries and keys).

Full grid: 6 targets × key-basis {unrotated, rotated} × select-basis {target, test} × budget {uniform, nonuniform} = 48.

Loading

import torch
sd = torch.load("hak_t256_kunrotated_starget_bnonuniform.pt", map_location="cpu")

Provenance

• Built with entrypoints.build_am_cache (repeat queries + --key/query/select-basis).
• Reconstruction validated with entrypoints.validate_ropefree_am --query-scheme repeat.
• Part of the Accordion / cartridges research project, branch rope_free_am_initialization, experiment group 0712_ropefree_am_sweep.

Headline result

Rope-free AM reconstructs far-query attention 3.5–8.5× worse than the true Strategy-B (rotated q + rotated k) baseline, and the gap widens with KV budget. The query basis is the dominant cost; KV budget and non-uniform allocation do not rescue rope-free. Best rope-free config ≈ unrotated·test·nonuniform at t512 (recon mean_rel 1.924); best overall in the grid is rotated·target·nonuniform·t256 (0.857).

These are research artifacts under a hard repeat/far-query, high-compression regime; relative axis comparisons are the signal.