Nanbeige4.1-3B-Gmail-Tool-Use

Nanbeige4.1-3B — Gmail Tool-Use (SFT + DPO)

Fine-tuned version of Nanbeige/Nanbeige4.1-3B for Gmail tool-calling tasks using a two-stage training pipeline.

<div align="center"> <img src="https://images.hdqwalls.com/wallpapers/king-glory-anime-boy-4k-ka.jpg" width="800" alt="Nanbeige Gmail Agent Chains" style="border-radius: 12px; box-shadow: 0 4px 12px rgba(0,0,0,0.2);"> <br><br> <h1>📧 Nanbeige-4.1-3B Gmail Tool Use Agent</h1> <p><i>A hyper-aligned 3B parameter agent matching GPT-4o-mini performance inside LangGraph.</i></p> </div> <br>

Training datasets: TurkishCodeMan/Nanbeige4.1-3B-Gmail-Tool-Use-Datasets

Training Pipeline

Stage 1 — Supervised Fine-Tuning (SFT)

• Dataset: 740 multi-turn Gmail agent traces (sft/traces_chatml_clean.jsonl)
• Format: ChatML with tool_calls (OpenAI function-calling schema)
• Method: LoRA r=16, α=32, 7 target modules
• Result: loss 0.8464 → 0.1888 · PPL 2.33 → 1.21

Stage 2 — Direct Preference Optimization (DPO)

• Dataset: 3223 preference pairs (dpo/dpo_dataset.jsonl) — 3 rejection strategies:
  • wrong_tool — incorrect tool selected (~34%)
  • missing_args — required arguments omitted (~32%)
  • bad_answer — poor final response (~34%)
• Method: DPO β=0.1, sigmoid loss, LoRA r=16, ref_model=None (PEFT implicit ref)
• Result: val_loss=0.000765 · reward accuracy=100% · normalized margin=+0.52

Supported Tools

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model = AutoModelForCausalLM.from_pretrained(
    "TurkishCodeMan/Nanbeige4.1-3B-Gmail-Tool-Use",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "TurkishCodeMan/Nanbeige4.1-3B-Gmail-Tool-Use",
    trust_remote_code=True,
)

Training Details