qwen2.5-coder-3b-synoema-tools-v2

Qwen2.5-Coder-3B — Synoema Tools v1

LoRA adapter fine-tuned on Qwen2.5-Coder-3B-Instruct for agentic tool-use with the Synoema MCP server.

Score: 92.9% (26/28 tasks) on the Synoema MCP agentic evaluation benchmark.

---

What is Synoema?

Synoema is an LLM-native programming language and runtime:

• BPE-aligned operators — every operator maps to exactly 1 cl100k_base token
• GBNF grammar for constrained decoding (structural correctness guarantee)
• Cranelift JIT + WebAssembly compilation targets
• MCP server exposing file_write, file_read, sno_typecheck, sno_run, search_corpus tools
• Contract annotations (requires/ensures) for formal verification

---

Model Details

---

Training Approach: Carousel Fine-tuning

This model was trained using a carousel strategy:

Base model → C7 (eval) → C2 (eval) → ... → C7 (best: 92.9%)
                                             ↑ always from best adapter

Each cycle:

1. Merge corpus — base corpus + all targeted examples for failing tasks
2. Train 3 epochs from the best previous adapter
3. Eval on 28 agentic tasks (real tool calls, real typecheck)
4. Analyze failures → generate targeted examples → add to corpus
5. Repeat from best adapter

---

Evaluation: 28 Agentic Tasks

The model is evaluated on real multi-turn tool-use scenarios. Each task requires calling MCP tools correctly in sequence. The eval runs actual sno typecheck and sno run commands — no mock results.

Result: 26/28 tasks passed (92.9%)

Remaining failures:

• TU4 — must write if x > y then x else y, see typecheck error, then fix to ? x > y -> x : y (2-write pattern)
• TU4, TU13 — same pattern with classify n = if n > 0 then 1 else 0 → ? n > 0 -> 1 : 0

Both require a strict write→typecheck→rewrite→typecheck sequence with exactly 2 file_write calls.

---

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-3B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-3B-Instruct")
model = PeftModel.from_pretrained(base, "Delimitter/qwen2.5-coder-3b-synoema-tools-v2")

With unsloth (recommended for inference):

from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Delimitter/qwen2.5-coder-3b-synoema-tools-v2",
    max_seq_length=1024,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

System prompt format (ChatML):

<|im_start|>system
You are an AI coding assistant for the Synoema programming language...
<|im_end|>
<|im_start|>user
Write a quicksort in Synoema to src/qs.sno and run it.
<|im_end|>
<|im_start|>assistant

---

Corpus Composition

All examples validated with sno check + sno run before training.

---

Training History (Carousel)

C7 was selected as best before catastrophic forgetting set in at C9.

---

Synoema Language Quick Reference

-- Ternary (no if/else!)
max x y = ? x > y -> x : y

-- Pattern matching
fact 0 = 1
fact n = n * fact (n - 1)

-- List comprehension  
evens = [x | x <- [1..20], x % 2 == 0]

-- Space-separated lists (NOT commas)
main = qsort [3 1 4 1 5 9]

-- ADT
Shape = Circle Int | Rect Int Int
area (Circle r) = 3 * r * r

---

License

Apache 2.0 — same as Qwen2.5-Coder base model.

Synoema is © Andrey Bubnov. See synoema.tech.