Downloads · 30 days
9
6% of all-time downloads
babylm2026-ipa
pakphum/babylm2026-ipa
babylm2026-ipa is a machine learning model from pakphum. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
LLaMA-style causal language model trained for BabyLM 2026 with IPA phonological feature augmentation.
All-time downloads
143
Public
Parameters
35.4M
142 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors142 MB · 98%
At a glance
- License
- mit
- Model type
- babylm_ipa
- Access
- Public
- Created
- Apr 3, 2026
- Updated
- Apr 4, 2026
- SHA
- 820fdd21
Model card
From the Hugging Face model README
babylm2026-ipa
LLaMA-style causal language model trained for BabyLM 2026 with IPA phonological feature augmentation.
Architecture
- Base: LLaMA-style transformer (RMSNorm, RoPE, SwiGLU FFN, weight-tied LM head)
- IPA variant:
ipa_full— gated fusion of panphon IPA feature vectors at the embedding layer - Hidden size: 512 | Layers: 6 | Heads: 8
- Vocab size: 32000 | Context: 512 tokens
- IPA dim: 24 (panphon binary articulatory features)
- Training split: 10M
IPA Fusion Variants
| Variant | Description |
|---|---|
baseline | Token embeddings only |
ipa_add | embed + W·ipa |
ipa_gate | embed + σ(W_g·ipa) ⊙ (W·ipa) |
ipa_full | Gated fusion + MSE auxiliary reconstruction loss |
Usage
from transformers import AutoConfig, AutoModelForCausalLM
import torch
config = AutoConfig.from_pretrained("pakphum/babylm2026-ipa", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("pakphum/babylm2026-ipa", trust_remote_code=True)
model.eval()
# input_ids from your tokenizer
input_ids = torch.tensor([[1, 2, 3, 4]])
# Optional: supply IPA vectors [B, T, ipa_dim=24]. Zeros = no phonological signal.
ipa_vectors = torch.zeros(1, 4, config.ipa_dim)
with torch.no_grad():
out = model(input_ids, ipa_vectors=ipa_vectors)
# out.logits: [B, T, vocab_size]
Citation
BabyLM 2026 shared task.
- Type
- babylm_ipa
- License
- mit
- Languages
- en, nl, zh
- Created
- Apr 3, 2026
- Updated
- Apr 4, 2026