stat214-lab3-bert-lora-r8-maxlen256

stat214-lab3-bert-lora-r8-maxlen256

LoRA adapter for bert-base-uncased, fine-tuned on transcripts from the Huth Lab fMRI story-listening dataset for the Stat 214 (Spring 2026) final project at UC Berkeley.

The adapter is used to extract context-aware word embeddings that are then fed into a per-voxel ridge regression to predict whole-brain BOLD signal from spoken-story stimuli.

Configuration

Encoding-model performance

After extracting per-word embeddings from this adapter (using ±10 word context windows + Lanczos downsampling + 4 TR delays) and fitting per-voxel ridge regression on Subjects 2 and 3:

(See full project repository for ridge weights, evaluation code, and SHAP / LIME word-importance analyses.)

Loading the adapter

from transformers import BertForMaskedLM, BertTokenizerFast
from peft import PeftModel

tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased")
base = BertForMaskedLM.from_pretrained("bert-base-uncased")
model = PeftModel.from_pretrained(base, "RheaTinghe/stat214-lab3-bert-lora-r8-maxlen256")
model.eval()

# Extract per-word embeddings via ±10 word context windows
# (see scripts/run_bert_pretrained.py in the project repo for the
# complete extraction pipeline)

Citation

@misc{stat214lab3,
  author = {Galloro, Drew and Wang, Ruihang and Khothsombath, Benjamin and Zhang, Rhea},
  title  = {Stat 214 Lab 3: BERT-LoRA encoding model for fMRI},
  year   = {2026},
  note   = {UC Berkeley Spring 2026},
}