var-lam

<h1 align="center">Variable-Array Reconstruction for Latent Acoustic Mapping</h1> <p align="center"> <img src="assets/img/logo_qmul.jpg" alt="QMUL" height="96"> </p> <p align="center"> <a href="https://philippxxy.github.io/var-lam/"> <img alt="Docs" src="https://img.shields.io/badge/docs-GitHub%20Pages-222222"> </a> <a href="https://github.com/PhilippXXY/var-lam"> <img alt="Source" src="https://img.shields.io/badge/source-GitHub-222222"> </a> <a href="https://github.com/PhilippXXY/var-lam/releases"> <img alt="Release" src="https://img.shields.io/github/v/release/PhilippXXY/var-lam"> </a> </p>

VAR-LAM is a research codebase for Variable-Array Reconstruction for Latent Acoustic Mapping. It predicts complex cross-spectral matrices (CSMs) from multi-channel audio by combining a learned STFT encoder, residual vector quantisation, microphone-pair reasoning, a factorised transformer, and a LAM-style steering-matrix decoder.

<p align="center"> <img src="docs/img/model_architecture.svg" alt="VAR-LAM model architecture" max-width="100%"> </p>

What This Repository Contains

• Checkpoints: published VAR-LAM baselines, ablations, direction-grid variants, and 4-to-32-channel upsampling models.
• Source: the complete implementation is maintained on GitHub.
• Documentation: setup, training, inference, and evaluation documentation is available at philippxxy.github.io/var-lam.

Setup

Install the source repository and its dependencies:

git clone https://github.com/PhilippXXY/var-lam.git
cd var-lam
uv sync --all-groups

The source repository contains small JSON pointers for published models. Its loader downloads checkpoints from this model repository and reuses the Hugging Face cache.

To download a standalone checkpoint explicitly:

hf download PhilippXXY/var-lam \
    checkpoints/32ch-to-32ch/ablations/components/transformer-off.pt \
    --revision main \
    --local-dir .

Checkpoints

Checkpoint paths are organised by channel mapping and experiment:

checkpoints/
├── 32ch-to-32ch/
│   ├── baseline.pt
│   ├── direction-grid/{64,128,256,512,1024,2048,4096,8192}.pt
│   ├── frequency/{linear-0hz-1khz,linear-0hz-4khz,mel-50hz-12khz}.pt
│   ├── input-representation/{magnitude-stft,partial-phat-stft,phat-stft}.pt
│   └── ablations/
│       ├── components/{cross-channel-attention-on,lstm-off-temporal-attention-off,lstm-off-token-comparison-off-transformer-off,token-comparison-no-diagonal,token-comparison-no-differences,token-comparison-no-geometry,token-comparison-no-products,token-comparison-off-transformer-off,token-comparison-off,transformer-off}.pt
│       ├── transformer-attention/{band-only,pair-only}.pt
│       └── rvq/{q2-k1024,q4-k1024,q4-k4096,q8-k64,q8-k256,q8-k512,q8-k2048,q12-k16,q16-k1024}.pt
├── 4ch-to-32ch/
│   ├── baseline.pt
│   ├── direction-grid/{64,128,256,512,1024,2048,4096,8192}.pt
│   ├── input-representation/{magnitude-stft,partial-phat-stft,phat-stft}.pt
│   └── ablations/
│       ├── components/{cross-channel-attention-on,lstm-off-token-comparison-off-transformer-off,rvq-off,token-comparison-off-transformer-off}.pt
│       └── microphone-layout/{horizontal,random,vertical-front}.pt
└── 4ch-to-4ch/
    ├── baseline.pt
    ├── input-representation/{magnitude-stft,partial-phat-stft,phat-stft}.pt
    └── ablations/components/{cross-channel-attention-on,lstm-off-temporal-attention-off,lstm-off-token-comparison-off-transformer-off,token-comparison-off-transformer-off,token-comparison-off,transformer-off}.pt

transformer-attention/ contains ablations isolated to the factorised transformer's attention axes. Optional modules and combined component ablations remain under components/. The two direction-grid/128.pt files are aliases of their corresponding baselines, whose decoder direction grid contains 128 points.

Training

Training code, configuration, and instructions live in the source repository. Training writes timestamped latest.pt and best.pt files locally; publish selected outputs here only when they are ready for reuse.

Inference

Run a pointer-backed checkpoint through the source repository:

uv run src/infer.py \
    --config configs/inference_config.yaml \
    --device cuda \
    --checkpoint checkpoints/32ch-to-32ch/ablations/components/transformer-off.pt \
    --checkpoint-revision main \
    --dataset locata

The loader reconstructs the architecture and channel-input mode from checkpoint metadata before loading the state dictionary.

Documentation

See the rendered VAR-LAM documentation for dataset setup, configuration, training, inference, metrics, and visualisation workflows.