URCHIN_BabyLM2026_Multilingual

URCHIN Multilingual

URCHIN (Unified Recurrent Connectome with Horizontal Integrate-and-fire Neurons) is a spiking, Dale-constrained recurrent language model for the BabyLM 2026 challenge (Multilingual track, 100M tokens (en/nl/zh, Byte-Premium adjusted)), built with the Parallelized Hierarchical Connectome Spiking State-space Model (PHCSSM) as its core architecture. A single cortex region of LIF neurons with a Dale-masked recurrent synapse is iterated K=24 lateral transmission steps per token to a fixed point; a linear head reads the cortex voltage. 4.23M parameters, no attention.

• Architecture: A single horizontal layer of Dale's-law excitatory/inhibitory LIF spiking neurons joined only by lateral (horizontal) connections: the same population receives the input current and is read out by a linear voltage head, with no separate input/output population; each token is resolved by a K-iteration fixed-point recurrence over the lateral synapse. No attention, no depth. Custom code (trust_remote_code=True).
• Track: BabyLM 2026 Multilingual (100M tokens (en/nl/zh, Byte-Premium adjusted)).
• Tokenizer: custom multilingual GPT-BERT tokenizer (16384; trained on the en/nl/zh corpus, not an official baseline).
• Serial (event-driven) variant: serial_urchin.py + configuration_serial_urchin.py provide an RSNN single-time-scan forward (UrchinSerialForCausalLM) reproducing the parallel outputs (score-equivalent), event-driven in O(T), using the SAME model.safetensors weights.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("multilingual", trust_remote_code=True).eval()
tok = AutoTokenizer.from_pretrained("multilingual")
ids = tok("The quick brown fox", return_tensors="pt").input_ids
with torch.no_grad():
    logits = model(ids).logits

Intermediate checkpoints

Intermediate training checkpoints are provided as git revisions named chck_<N>M for the BabyLM challenge fast-eval.

License and citation

Released under CC BY-NC 4.0 (attribution required, non-commercial only). If you use this model or code, please cite (see CITATION.cff):

@misc{anonymous2026urchin,
  title        = {URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining},
  author       = {Anonymous},
  year         = {2026},
  howpublished = {under review (anonymized)},
  note         = {URCHIN spiking recurrent language model, BabyLM 2026}
}

Provenance and integrity fingerprints (canary + weight SHA-256) are documented in PROVENANCE.md.