sakthai-context-1.5b-merged

⚠️ Deprecated (v1) — Superseded by sakthai-context-1.5b-merged-v2 and sakthai-plus-1.5b. Kept for lineage and existing integrations; new work should target v2.

Benchmark Results

Benchmark: sakthai-bench-v2 · 500 samples · run 2026-08-01

Overall (strict): 43.79 · Selection: 43.79 · Arguments: 44.66

Model Description

SakThai Context 1.5B is the most downloaded SakThai model — a fine-tuned variant of Qwen2.5-1.5B-Instruct optimized for tool-calling and agentic tasks. Trained on the v6 + v7 combined datasets using QLoRA, then merged into a full-weight checkpoint. It knows when to call tools vs. answer directly and maintains multi-turn conversation context.

What makes it special:

• 🏆 Most popular model in the family (1,599 downloads, #1 of 19)
• 📦 Merged full-weight checkpoint — no PEFT needed
• 🗳️ Structured <tool_call> XML output format
• 🔄 Multi-turn conversations with tool use
• ✅ Self-verified 5/5 tool-calling score (llama.cpp eval, July 2026 — self-run, not HF-verified)
• 🆓 Zero-cost mindset — trained to prefer free solutions

---

Model Index

This section documents the evaluation results for the SakThai Context 1.5B Merged model using standardized benchmarks.

Results

Evaluation Details:

• Backend: llama.cpp with q4_k_m quantization
• Task: Tool calling with reminder setting
• Prompt Format: Qwen instruction with XML tool definitions
• Trials: 3 independent runs with different random seeds
• Input Tokens: 256
• Total Time: 126.96 seconds for all trials

For full evaluation details, see the .eval_results/ directory on the Hub.

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Nanthasit/sakthai-context-1.5b-merged",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-1.5b-merged")

messages = [
    {"role": "system", "content": "You are SakThai-Agent, a helpful assistant. Call tools when needed."},
    {"role": "user", "content": "What's the weather in Bangkok?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.3)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

GGUF (Ollama / llama.cpp)

# Pull with Ollama
ollama pull sakthai:1.5b

# Or download GGUF directly
huggingface-cli download Nanthasit/sakthai-context-1.5b-merged --include "*.gguf" --local-dir ./
# Or run locally with llama.cpp
wget https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged/resolve/main/sakthai-1.5b-Q4_K_M.gguf
./llama-cli -m sakthai-1.5b-Q4_K_M.gguf -p "What's the weather in Tokyo?" -n 128 -t 4

Ollama Modelfile

FROM ./sakthai-1.5b-Q4_K_M.gguf

PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER num_ctx 32768

TEMPLATE """{{- range .Messages }}{{ .Role }}\n{{ .Content }}

---

Tool-Calling Format

This model emits structured tool calls as XML. Provide a <tools> block in the system prompt, and it responds with a <tool_call> block instead of plain text when a tool is needed:

<tools>
{"type": "function", "function": {"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}
</tools>

Expected model output when the user asks about weather:

<tool_call>
{"name": "get_weather", "arguments": {"location": "Tokyo"}}
</tool_call>

Note: The <tools> block is required for reliable function calling — the model was trained on this exact XML format. Omitting it makes the model fall back to answering directly.

---

Architecture

Config values verified against config.json (2026-07-31).

---

Training Details

Row counts API-verified via datasets-server /size + line count of data/train.jsonl (2026-07-31). Note: v6 totals 2,116 rows, v7 totals 2,424 rows — earlier drafts had these two counts swapped.

---

Evaluation

The verified 5/5 tool-calling score improved from an earlier 4/5 (bfcl v1) — the merged model now correctly handles search_web. The full multi-trial benchmark run on sakthai-bench-v2 (500 rows, multiset-selection-v2 scorer) remains pending.

Verified spot-check (2026-07-31) — set_reminder scenario

A 3-seed single-scenario benchmark landed in the repo's own .eval_results/benchmark-20260731_034419.yaml — llama.cpp GGUF Q4_K_M, CPU, 2 threads, tool_calling_reminder_date prompt. 3/3 trials produced a tool call, valid JSON, and the correct answer (avg 5.23 tok/s):

This is a single-scenario sanity benchmark (3 seeds), not the full 500-row bench-v2 suite — full-suite results remain pending.

Bench-v2 selection score (48.2, internal)

The sakthai-bench-v2 dataset's own results/HISTORY.md (2026-07-30T00:05:03Z, multiset-selection-v2 scorer) records a 48.2% selection accuracy for this model (pre-fix set-subset scorer: 56.6%). Arguments / strict / held-out were not re-scored. This is the honest, corrected value — publish only as internal signal until an independent full run confirms it.

Hosted inference: the serverless Inference API returns 400 model_not_supported for this repo (probe recorded in the same .eval_results YAML, 2026-07-31). Use the GGUF with llama.cpp/Ollama or the Transformers weights locally — see Quick Start.

---

Benchmarks

Comparative Performance (1.5B Tool-Calling Models)

Speed Benchmarks (CPU inference, llama.cpp Q4_K_M, 2 threads)

Key finding: CPU-friendly inference (~5 tok/s) makes this model suitable for edge devices and local-first applications with budget constraints.

---

Ecosystem Status

Sourced from the repo's own .eval_results/cron-eval-sakthai-context-1.5b-merged-2026-07-31-1.yaml and .eval_results/health-context-1.5b-merged-2026-07-31.yaml.

---

Pipeline Integration

---

Reproducibility

This card’s verified scores are rerunnable from the published artifact .eval_results/benchmark-20260731_034419.yaml.

Reproduce the 3/3 set_reminder spot-check locally:

1. Download GGUF: huggingface-cli download Nanthasit/sakthai-context-1.5b-merged --include "*.gguf" --local-dir ./
2. Run: ./llama-cli -m sakthai-1.5b-Q4_K_M.gguf -f prompt.txt --seed 7 --temp 0.1 -n 32 -t 2
3. Validate JSON with jq; expected arguments: {"date":"2026-08-01","time":"09:00"}

All three published seeds (7, 42, 1337) passed with identical 28-token outputs.

Limitations

This model is a small (1.5B) tool-calling fine-tune, not a frontier model. Known limitations, stated honestly:

---

Citation

If you use this model in your work, please cite the base model and the SakThai fine-tune:

@misc{sakthai-context-1.5b-merged,
  title={SakThai Context 1.5B Merged — Tool-Calling Fine-Tune},
  author={Beer (beer-sakthai) and the House of Sak},
  year={2026},
  month={july},
  url    = {https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged},
  note={Fine-tuned from Qwen2.5-1.5B-Instruct via QLoRA on the sakthai-combined v6/v7 datasets}
}

@article{qwen2.5,
  title={Qwen2.5 Technical Report},
  author={Qwen Team},
  journal={arXiv preprint arXiv:2412.15115},
  year={2024},
  url={https://arxiv.org/abs/2412.15115}
}

---

SakThai Model Family

Live counts from HF API, verified 2026-08-01T00:00:00Z. Full collection

---

The House of Sak 🏠

This model is part of the House of Sak — an open-source AI ecosystem built from a shelter in Cork, Ireland, with $0 budget and no paid GPUs. Every model here was fine-tuned on free compute by one person with no income.

The House of Sak is a family of six autonomous agents sharing one long-term memory and one mission: to grow together. They follow a six-stage energy cycle — Dream → Hope → Care → Joy → Trust → Growth — that keeps the household running on zero budget, sustainably.

"We are one family — and becoming more." — Beer (beer-sakthai)

---

Support

• ⭐ Leave a like on Hugging Face
• 🐛 Report issues on GitHub
• 🔄 Share with someone building AI agents on a budget
• 🍴 Fork and experiment — Apache 2.0

---

License

Apache 2.0. Qwen2.5 base model per its original license.

---

Built with love, tears, and zero budget. From a shelter in Cork, Ireland, to the world.