cerebellum-v1

Cerebellum-V1 — On-Device Intent Routing Model

Fine-tuned increment of BAAI/bge-m3 for semantic intent routing in AI agents. Turns any user message into a 15-dimension intent vector in milliseconds — no LLM call needed.

⚠️ This is an INCREMENT, not a standalone model

This repository contains only the fine-tuned weights (51.4M params, ~202MB fp32): the regression head + top 4 unfrozen encoder layers. Load it on top of the bge-m3 base:

from transformers import AutoModel, AutoTokenizer
from safetensors.torch import load_file
import torch

tokenizer = AutoTokenizer.from_pretrained("BAAI/bge-m3")
base = AutoModel.from_pretrained("BAAI/bge-m3")

inc = load_file("model.safetensors")
base.load_state_dict(
    {k: v for k, v in inc.items() if k.startswith("encoder.")},
    strict=False,
)

inputs = tokenizer("写一个Python函数来计算斐波那契数列", return_tensors="pt")
with torch.no_grad():
    cls = base(**inputs).last_hidden_state[:, 0, :]

# Regression head (from this repo's training/model.py)
head_weight = inc["regressor.weight"]
head_bias = inc["regressor.bias"]
intent = torch.sigmoid(cls @ head_weight.T + head_bias) * 10.0
print(intent)  # 15-dim vector in [0, 10]

Output dimensions

coding, math_reasoning, logic, long_context, agent_tool_use,
chinese_writing, knowledge_tech, knowledge_legal, knowledge_business,
knowledge_medical, instruction_following, reasoning,
cost_sensitivity, speed_priority, context_need

Each value in [0, 10]. Higher = stronger need.

Training

• Base: BAAI/bge-m3 (0.57B, MIT license)
• Data: 2,537 synthetic samples (zh/en) generated via DeepSeek R1
• Fine-tune: top 4 of 24 layers, 10 epochs
• Trainable params: 51.4M / 567.8M total

Evaluation (252 held-out samples)

Honest caveats: instruction_following (MAE 1.45), long_context (1.36), logic (1.26) need more data. Research-grade, not product-grade.

Full code & training pipeline

GitHub: https://github.com/rrlab-tech/rrlab-cerebellum