assistllm-v0

Assist-LLM v0

<div align="center"> <b>A ~150M-parameter tool-calling model, trained from scratch for less than the cost of a meal.</b><br> _{DeepSeek-V3-style MLA + MoE · no thinking mode · 337MB · runs on a phone · total GPU bill ≈ 2.7 burgers} </div> <hr>

Assist-LLM v0 is a tiny agentic model built for Page Assist — it chats, and more importantly it calls tools: given function schemas, it decides whether to call, extracts arguments as JSON, stops cleanly, and grounds its answer in the tool's response.

It was trained from a random initialization on a single rented RTX 5090 — pretraining, SFT, calibration, evals and all — for ~$14 total. The entire pipeline is open: data manifest → tokenizer → pretrain → SFT → calibration → benchmarks. Knowledge cutoff: early 2026 (it will tell you this itself if asked).

Benchmarks (0-shot)

All via lm-evaluation-harness, 0-shot; When2Call via NVIDIA's MCQ configs adapted to this model's chat format.

Read these numbers honestly: this is a 150M model pretrained on 800M tokens (models like SmolLM2-135M use 2T). General-knowledge MCQ is its weakest suit — it is not what the model is for. The model's job is the tool-calling protocol, and there it performs:

The bill (in burgers)

Pretrain 3.1h + SFT 6h + calibration ~0.6h + evals ~1h on one RTX 5090 at $0.524/h, plus a few false starts on slower boxes. Frontier labs measure training runs in GDP; this one is lunch.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("n4ze3m/assistllm-v0")
model = AutoModelForCausalLM.from_pretrained("n4ze3m/assistllm-v0", dtype=torch.bfloat16, device_map="cuda")

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {"type": "object",
                       "properties": {"city": {"type": "string"}},
                       "required": ["city"]},
    },
}]

messages = [{"role": "user", "content": "weather in Kochi?"}]
enc = tok.apply_chat_template(messages, tools=tools, add_generation_prompt=True,
                              tokenize=True, return_dict=True, return_tensors="pt").to("cuda")
out = model.generate(**enc, max_new_tokens=80, do_sample=False,
                     eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=False))

<tool_call>
{"name": "get_weather", "arguments": {"city": "Kochi"}}
</tool_call><|im_end|>

Feed the tool result back as a {"role": "tool", "content": ...} message and it answers from it.

GGUF (llama.cpp) — experimental

Shipped in this repo, quantized from the bf16 weights:

llama-cli -m assistllm-v0-q4_k_m.gguf -no-cnv

⚠️ Experimental: the GGUF path (llama.cpp/Ollama) chats fine and keeps the identity, but the tool-call trigger is less reliable than the transformers path — llama.cpp's DeepSeek-MLA implementation assumes the flagship's head dims, and this model's custom dims (10 heads, nope/rope 96/32) sit slightly off that math. For dependable tool calling use the safetensors weights (above). An Ollama Modelfile with the correct ChatML Go-template is included at ollama/Modelfile.

Architecture

DeepSeek-V3-style, scaled to pocket size — 168M total / ~80M active parameters:

• MLA (Multi-head Latent Attention) — compressed KV cache, cheap long context
• DeepSeekMoE — fine-grained experts, most parameters asleep per token
• 32k BPE tokenizer (trained on the pretrain corpus), 4096 context
• ChatML-style format with <tools> / <tool_call> / <tool_response>; assistant-only loss via {% generation %} blocks
• No thinking mode — it just answers

Training

Stage D exists because v0.9 over-triggered: with tools in context it called them for everything, echoing the schema ({"city": "city"}) when it couldn't extract an argument — the exact pathology NVIDIA's When2Call work describes ("public datasets contain many examples of calling tools but far fewer of not calling them"). One calibration pass later: judgment fixed, identity learned, social chit-chat still trigger-happy (see below).

Known limitations

• Weak general knowledge, math, and prose. 150M params × 800M tokens — give it tools, that's the point.
• Social over-triggering: "hi" / "thanks" with tools in context may produce a spurious call with a placeholder argument. App-layer guard: reject any argument value that literally equals its schema property name and treat the turn as chat.
• English-only, text-only, no vision.
• This is a research artifact, not a production assistant. Verify anything that matters.

License

Model weights and code: Apache 2.0. Training data remains under its original datasets' licenses.

Use of LLMs

We leverage large language models (LLMs) to assist in drafting and polishing written content. Specifically, LLMs are used to improve clarity, coherence, and readability of the text, while ensuring that technical accuracy and intended meaning are preserved. All outputs generated by the LLM are carefully reviewed and edited by the authors to maintain factual correctness and align with the scientific content. The LLM serves as a tool to support writing efficiency, not to generate original research ideas or conclusions.

Citation

@misc{assistllm-v0,
  title  = {Assist-LLM v0: a 150M tool-calling model trained for less than a meal},
  author = {Muhammed Nazeem (n4ze3m), Page Assist},
  year   = {2026},
  url    = {https://huggingface.co/n4ze3m/assistllm-v0}
}