Eigenbrain

Eigenbrain: frozen brain-decoding inference bundle

This repository contains frozen inference assets for three independent brain-decoding demonstrations: EEG classification, neural language decoding, and NSD Subject-1 fMRI-to-image reconstruction. It is an inference/replay bundle; it does not train a model and does not fit preprocessing transforms.

Contents

The language route uses the public base models microsoft/Phi-4-mini-instruct and BAAI/bge-m3; the files in this repository are the project-specific alignment, projector, and LoRA weights. The image route uses a large frozen unCLIP checkpoint and requires a CUDA GPU with substantial storage and memory.

Download

Download this repository together with the companion inference code. The project code expects the following local layout:

assets/
weights/
weights/alice_lora/
weights/nsd_sub01/
models/Phi-4-mini-instruct/
models/bge-m3/

The companion download_assets.py downloads the project files from this repository and the two public base models:

export EIGEN_HF_REPO=SSp1ash/Eigenbrain
python download_assets.py

For classification and language only, omit the large NSD image files with python download_assets.py --skip-nsd.

Run

python run_demo.py --task classification --device cuda

python run_demo.py --task language --device cuda \
  --phi-path models/Phi-4-mini-instruct \
  --bge-path models/bge-m3

python run_demo.py --task image --device cuda

The image command evaluates the default ten held-out NSD examples and writes the selected qualitative reconstructions and image-level metrics under outputs/nsd_sub01/.

Reference results

The fixed compact replay gives approximately 40% accuracy for EEG classification on 10 examples. The language text route gives a mean BGE sentence cosine of approximately 0.81 on 5 examples. The NSD image route has an archived full-test CLIP 2-way identification score of 0.783; the default ten-example smoke test is not a replacement for the full benchmark.

These values describe the shipped frozen replay assets and should not be interpreted as a new independent held-out evaluation. The language alignment checkpoint includes historical oracle-style provenance, and the NSD visual demo saves only a small qualitative subset of the evaluated examples.

Files

Custom project files are stored under assets/ and weights/. The largest file is weights/nsd_sub01/unclip6_epoch0_step110000.ckpt (about 18 GB). Public base models are intentionally not duplicated here.

License and limitations

The code and frozen assets are provided for research demonstration and reproducibility. Check the licenses and terms of the public Phi-4, BGE-M3, MindEye2/SDXL-unCLIP components and the NSD data before redistribution or commercial use. The NSD route is within-subject reconstruction for Subject 1, not a cross-subject general-purpose image decoder.