nesso2-0.4B-agentic

Nesso2-0.4B-Agentic

Nesso2-0.4B-Agentic is a bilingual Italian/English Small Language Model (SLM) optimized for function calling, structured output generation, and multi-step agentic execution, with a deliberate emphasis on Italian tool use. It is post-trained on top of a knowledge-enriched, long-context checkpoint of Zagreus-0.4B-ita — a foundational model trained from scratch by the mii-llm community (Made in Italy – Large Language Model) on the Seeweb HPC infrastructure.

Designed for sovereign edge inference, Nesso2-0.4B-Agentic targets deployment scenarios that require reliable tool use, structured JSON output, correct tool vs. no-tool discrimination, and multi-step agentic reasoning — all within a compact ~0.4B parameter footprint and a 32k-token context window.

On our bilingual function-calling benchmark it is, to our knowledge, the strongest open SLM for Italian agentic tool use in its size class, edging out Qwen3-0.6B overall and beating it by a wide margin on Italian.

⚠️ This model is at the SFT (Supervised Fine-Tuning) stage. DPO (Direct Preference Optimization) is planned; updated results will be published upon completion.

---

Model Details

---

Lineage

Unlike a single-stage SFT model, Nesso2-0.4B-Agentic is the tip of a multi-stage pipeline designed to give a tiny model both knowledge and agentic skill:

zagreus-0.4B-ita  (base, pre-trained from scratch, ~1T tokens)
        │
        ▼
  + Knowledge CPT           continued pre-training on a curated knowledge
        │                   corpus (Italian/English Wikipedia, augmented QA)
        ▼                   to lift factual/MMLU capability
  + 32k Context Extension   long-context adaptation (RoPE θ = 1e6)
        │
        ▼
  + Agentic SFT (v8)        supervised fine-tuning on bilingual instruction +
                            function-calling data (this model)

The knowledge CPT stage is what separates this model from a plain SFT on the same base: it measurably improves factual benchmarks (see Evaluation) and is the reason Italian MMLU/ARC hold up despite the heavy agentic specialization.

---

Training Details

Base Model Pre-training

The foundation, Zagreus-0.4B-ita, was pre-trained on approximately 1 trillion tokens:

Token distribution: ~400B English + ~400B Italian + ~200B Code Infrastructure: 64× NVIDIA A100 (8 nodes × 8 GPUs) on Seeweb HPC Framework: Nanotron (mii-llm fork)

Knowledge CPT + 32k Context Extension

Continued pre-training (Nanotron) on a curated knowledge corpus to break the small-model MMLU ceiling, followed by a long-context extension stage to 32,768 tokens (RoPE θ raised to 1e6). Run on the Seeweb HPC infrastructure.

Post-training (Agentic SFT)

Supervised fine-tuning with TRL (SFTTrainer) + FSDP, on the Seeweb HPC infrastructure.

The instruction dataset is a bilingual (Italian/English) mixture combining broad conversational/instruction data with a synthetic function-calling corpus covering single- and parallel tool calls, argument disambiguation, missing-argument handling, unavailable-tool refusal, observation grounding, multi-step trajectories, and — critically — no-tool discrimination (answering directly when a tempting tool is present but unnecessary). Tool schemas and argument names are randomized to discourage memorization.

Key hyperparameters:

---

Chat Template

This model uses the Llama-3 conversation format (not ChatML). Tools are provided through the tools argument of apply_chat_template, and the model emits calls as Hermes-style <tool_call> blocks.

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful assistant with access to tools.<|eot_id|><|start_header_id|>user<|end_header_id|>

What is the weather in Rome today?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

A tool call looks like:

<tool_call>
{"name": "get_weather", "arguments": {"city": "Roma"}}
</tool_call>

Special tokens:

• bos_token: <|begin_of_text|> (128000)
• eos_token: <|eot_id|> (128009)
• pad_token: <|finetune_right_pad_id|> (128004)

⚠️ The saved inference template does not emit the BOS token itself — tokenize the rendered string with add_special_tokens=True (as shown below) so that exactly one BOS is prepended. Do not double-add it.

---

Usage

Function calling

import re, torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "mii-llm/nesso2-0.4B-agentic"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
).eval()

def chat(messages, tools=None, max_new_tokens=256):
    # Render with the Llama-3 tool template, then tokenize adding exactly one BOS.
    prompt = tokenizer.apply_chat_template(
        messages, tools=tools, tokenize=False, add_generation_prompt=True
    )
    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=True).to(model.device)
    n = inputs["input_ids"].shape[1]

    out = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=False,                 # PURE greedy — best for structured tool calls
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )
    text = tokenizer.decode(out[0][n:], skip_special_tokens=False)
    answer = re.split(r"<\|eot_id\|>|<\|end_of_text\|>", text)[0].strip()

    calls = re.findall(r"<tool_call>\s*(\{.*?\})\s*</tool_call>", answer, flags=re.S)
    return answer, calls

# A small toolbox the assistant can choose from
tools = [
    {"type": "function", "function": {
        "name": "get_weather", "description": "Meteo attuale per una città",
        "parameters": {"type": "object",
            "properties": {"city": {"type": "string"}}, "required": ["city"]}}},
    {"type": "function", "function": {
        "name": "convert_currency", "description": "Converte un importo tra due valute",
        "parameters": {"type": "object",
            "properties": {"amount": {"type": "number"}, "from": {"type": "string"}, "to": {"type": "string"}},
            "required": ["amount", "from", "to"]}}},
    {"type": "function", "function": {
        "name": "search_restaurants", "description": "Cerca ristoranti in una città",
        "parameters": {"type": "object",
            "properties": {"city": {"type": "string"}, "cuisine": {"type": "string"}}, "required": ["city"]}}},
]
SYSTEM = "Sei un assistente che può usare strumenti quando servono."

# 1) Selects the right tool among several and fills the arguments
_, calls = chat([{"role": "system", "content": SYSTEM},
                 {"role": "user", "content": "Quanto sono 100 euro in dollari?"}], tools=tools)
print(calls)
# -> ['{"name": "convert_currency", "arguments": {"amount": 100, "from": "EUR", "to": "USD"}}']

# 2) Emits parallel calls for a multi-part request
_, calls = chat([{"role": "system", "content": SYSTEM},
                 {"role": "user", "content": "Che tempo fa a Roma e a Torino?"}], tools=tools)
print(calls)
# -> ['{"name": "get_weather", "arguments": {"city": "Roma"}}',
#     '{"name": "get_weather", "arguments": {"city": "Torino"}}']

ℹ️ Give complete requests. The model is strongest when the required arguments are present in the user's message (it excels at tool selection and parallel calls). Like most models this size, when a required argument is missing it may fill a sensible default instead of always asking for it — so validate required arguments before executing a call.

💡 Tip: For function calling and structured output, use pure greedy decoding (do_sample=False, no repetition penalty). The tool-call JSON is short and the prompt template already contains the structural tokens (name, arguments, quotes, braces) — a repetition penalty or no_repeat_ngram_size will suppress exactly those tokens and corrupt the JSON (e.g. emitting "Name" or dropping arguments). For long free-form conversation, a light repetition_penalty (≈1.15) can help avoid loops on a model this small, but keep it off for tool calls.

Plain conversation (no tools)

For free-form chat, drop the tools argument and add a light repetition penalty to keep a model this small from looping. The same tokenizer/model loaded above are reused.

def chat_plain(messages, max_new_tokens=256):
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=True).to(model.device)
    n = inputs["input_ids"].shape[1]

    out = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=False,
        repetition_penalty=1.15,         # light penalty helps free-form text (NOT for tool calls)
        no_repeat_ngram_size=6,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )
    text = tokenizer.decode(out[0][n:], skip_special_tokens=False)
    return re.split(r"<\|eot_id\|>|<\|end_of_text\|>", text)[0].strip()

# Single turn
messages = [
    {"role": "system", "content": "Sei un assistente utile e conciso."},
    {"role": "user", "content": "Spiegami in due frasi cosa è il machine learning."},
]
reply = chat_plain(messages)
print(reply)

# Multi-turn: append the assistant reply and the next user turn, then call again.
messages.append({"role": "assistant", "content": reply})
messages.append({"role": "user", "content": "Fammi un esempio concreto."})
print(chat_plain(messages))

Example output for the single-turn call:

Il machine learning (ML) è una branca dell'intelligenza artificiale che permette ai
computer di apprendere dai dati, migliorando le proprie prestazioni senza essere
esplicitamente programmati per un compito specifico.

---

Production notes & recommendations

Nesso2-0.4B-agentic (v8) is the recommended release model — it is the best Italian tool-caller and the best Italian conversationalist of its family, and its Italian lead over Qwen3-0.6B is confirmed on two independently-authored function-calling benchmarks (see Evaluation). For a single Italian deployment it serves both agentic and conversational use.

Because it is a ~0.4B model, two lightweight application-layer wrappers make it robust in production:

1. Validate arguments before executing a call. Like most models this size, when a required argument is missing the model may fill a plausible default instead of asking. Before executing any tool call, verify each required-argument value is actually grounded in the user's message; if not, treat it as missing and ask the user. This prevents acting on hallucinated values (e.g. emailing an invented recipient).

2. Feed observations back explicitly. After a tool returns, pass the result to the model as a tool turn and ask for the final answer. If the model tries to re-call the tool instead of answering from the observation, short-circuit and prompt it to answer using the returned data. Observation grounding is the model's relative weak spot (see Limitations), and this wrapper neutralizes it.

Decoding: pure greedy for tool calls (a repetition penalty corrupts the JSON); a light repetition_penalty ≈ 1.15 is fine for free-form chat.

---

Evaluation

Three complementary evaluation families were used. Academic benchmarks were run with our fork of lm-evaluation-harness; agentic and conversational quality were measured with dedicated bilingual test suites.

1. Academic benchmarks

MMLU is 5-shot acc; HellaSwag / ARC are 0-shot acc_norm; IFEval is inst_level_loose_acc (generative, chat template). All numbers are on a 0–1 scale. All models were run with the same script; in particular IFEval uses each model's chat template. (An earlier version of this card mixed protocols — Qwen3-0.6B IFEval without the chat template, an Italian ARC typo and an English MMLU of a different setting — which understated Qwen; the figures below are the corrected, like-for-like ones.)

Italian

English

Overall

Takeaways. On academics, Qwen3-0.6B is ahead overall (0.3955 vs 0.3534). Nesso2-0.4B-agentic leads it on Italian HellaSwag (+4.8) and Italian ARC (+3.6) and is level on English HellaSwag — the knowledge CPT stage keeps the Italian commonsense/reasoning gains that similarly-sized SLMs usually lack. It also outperforms its sibling nesso-0.4B-agentic on MMLU in both languages (Italian 0.326 vs 0.282; English 0.336 vs 0.294), which is the CPT stage paying off. Qwen is clearly ahead on MMLU (+7.8 Italian, +13.7 English: a knowledge gap favoring its far larger pre-training budget), on IFEval (+8.1 Italian, +8.5 English) and on English ARC. Nesso2's edge over Qwen3-0.6B is in agentic tool use and Italian conversation (below), not on academic leaderboards.

2. Agentic function calling (bilingual, 100 cases) ⭐

Our frozen function-calling suite: 100 bilingual cases across 10 categories (single / parallel-same / parallel-different tool calls, multi-argument, disambiguation, missing-argument, unavailable-tool refusal, no-tool discrimination, observation grounding, multi-step). Greedy decoding, Hermes <tool_call> format, per-category automatic scoring.

Nesso2-0.4B-agentic is best overall and decisively ahead on Italian tool use (+6), while Qwen keeps an English advantage. This is the benchmark the model is optimized for, and where its real-world value over general-purpose SLMs shows.

Speed matters for agents — the scoped claim. Agentic tool-calling is latency-sensitive, so the relevant comparison is the fast, single-forward-pass regime. On an independent function-calling benchmark, Nesso2 leads Italian FC over Qwen3-0.6B's non-thinking mode (63.1% vs 51.3%) at the same speed — ~0.83 s / ~40 tokens per call. Qwen's thinking mode reaches higher raw accuracy (Italian FC 73.1%) but at ~6× the latency (4.71 s) and ~5× the tokens — a different latency class for real-time use. Even against thinking-Qwen, Nesso2 still wins multi-step (67–70 vs 14) and parallel-same-tool (91 vs 76). So: the best Italian tool-caller at low latency / without test-time reasoning, and ~6× faster than the reasoning alternative.

3. Conversational quality (LLM-as-judge)

20 bilingual multi-turn tasks per language, graded 1–10 by Qwen3.6-35B-A3B on correctness / language-fidelity / helpfulness (greedy answers). Mean overall score:

Despite its agentic specialization, Nesso2-0.4B-agentic delivers the best Italian conversational quality of its lineage (4.40, tied with nesso-0.4B-agentic) and strongly outscores Qwen3-0.6B in Italian chat (4.40 vs 2.80). English conversation remains its relative weak spot.

Discussion

Nesso2-0.4B-agentic is a task-specialized model: its post-training prioritizes structured-output fidelity, tool-calling accuracy, no-tool discrimination, and agentic planning. Thanks to the knowledge-CPT stage, this specialization comes without giving up the CPT knowledge gains — it keeps its lead over Qwen3-0.6B on Italian HellaSwag/ARC and beats it on the agentic suite, while Qwen remains ahead on MMLU and IFEval, while remaining a genuinely useful Italian conversationalist. Its edge over general-purpose SLMs of similar size is best assessed on agentic and function-calling tasks, not academic leaderboards.

Limitations

Known weak spots, all manageable with the wrappers in Production notes:

• Observation grounding. After a tool returns a result, the model sometimes answers from its own priors or re-calls the tool instead of grounding on the returned data. Mitigate by feeding observations back explicitly (wrapper 2).
• Abstention on tempting cases. When a required argument is missing or no tool applies, the model is reliable on many phrasings but can fire a tool (or fill a default value) on borderline, tool-tempting prompts. Mitigate with argument-grounding validation (wrapper 1). This is phrasing-sensitive, not uniform.
• English < Italian. English tool use, English chat, and English MMLU trail Qwen3-0.6B. This is a deliberate Italian-first trade — for an English-primary deployment, Qwen3-0.6B is the stronger pick.
• ~0.4B capacity limits. Different-tool parallel calls, multi-argument calls, and exact multi-step completion are capacity-bound and do not reach large-model reliability; keep such flows simple or supervised.

(Two later experiments confirmed these are the real edges: a grounding-heavy variant fixed observation handling but regressed conversation, and a weight-merge that topped one benchmark did not hold up on a stricter independent one. Nesso2 = v8 remains the balanced release. See the nesso2 README for the full record.)

---

Related Models

---

Citation

If you use this model in your research, please cite:

@misc{zagreus2025,
  title        = {The Joy and Pain of Training an LLM from Scratch:
                  A Technical Report on the Zagreus and Nesso Model Families},
  author       = {mii-llm community},
  year         = {2025},
  howpublished = {\url{https://github.com/mii-llm/zagreus-nesso-slm}},
}

---

Acknowledgements

• Antonio Baldassarra (CEO, Seeweb) and Marco Cristofanilli (Head of AI, Seeweb) for infrastructure sponsorship
• The Hugging Face team for Nanotron, datatrove, FineWeb, and FineWeb-2
• The mii-llm open-source community

---

License

Released under the Apache 2.0 license.

Made with ❤️ in Italy by mii-llm