Agents-A1-4B-kimi-Preview

<h1 align="center">🤖 Agents-A1-4B-kimi-Preview</h1> <p align="center"> <b>4B Coding Agent Model — SFT Fine-tuned · Kimi K3 Traces · Tool-Calling</b> </p> <p align="center"> <img src="https://img.shields.io/badge/license-AGPL--3.0-red"> <img src="https://img.shields.io/badge/Agents--A1-4B-blue"> <img src="https://img.shields.io/badge/SFT-LoRA-brightgreen"> <img src="https://img.shields.io/badge/Kimi--K3-Traces-orange"> <img src="https://img.shields.io/badge/Preview-v0.1-yellow"> </p> <br>

Preview release — Fine-tuned from InternScience/Agents-A1-4B on Kimi K3 coding and debugging traces. Specializes in coding agent behavior, debugging, and step-by-step tool-use.

---

✨ Key Features

---

🚀 Usage

Python (Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "hotdogs/Agents-A1-4B-kimi-Preview",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("hotdogs/Agents-A1-4B-kimi-Preview")

messages = [{"role": "user", "content": "Write a Python function to sort a list"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
print(tokenizer.decode(outputs[0]))

GGUF (llama.cpp)

GGUF quantized version coming soon.

---

📦 Downloads

---

🧬 Architecture

---

🏋️ Training Details

Base Model

InternScience/Agents-A1-4B — A 4B VLM based on Qwen3.5 architecture with hybrid linear/full attention. Fine-tuned on the text decoder only.

Dataset

greghavens/kimi-k3-coding-and-debugging-traces — 4,928 high-quality coding agent traces from Kimi K3, featuring:

• Real coding and debugging trajectories
• Multi-step tool-use patterns
• 156 task categories (build-game, dependency-planning, debug-concurrency, etc.)
• Languages: Python, C, C++, Go, Java, Rust, Bash, etc.
• Auto-generated coding agent reasoning

Preprocessed: 3,389 clean rows after deduplication and filtering (3,219 train / 170 val).

Training Pipeline

┌─────────────────────────────────────────────────────────────┐
│                    Training Pipeline                         │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  Base Model                     LoRA Adapter                │
│  (Agents-A1-4B)                 (r=64, α=128)              │
│       │                              │                     │
│       └───────────┬──────────────────┘                     │
│                   │                                        │
│         Unsloth FastLanguageModel                           │
│         device_map="auto" · 4× RTX 3060 12GB               │
│                   │                                        │
│                   ▼                                        │
│    ┌──────────────────────────┐                            │
│    │  Custom SFTDataset        │                            │
│    │  • apply_chat_template()  │                            │
│    │  • Tokenize (4096 max)    │                            │
│    │  • Mask prompt tokens     │                            │
│    └──────────┬───────────────┘                            │
│               │                                            │
│               ▼                                            │
│    ┌──────────────────────────┐                            │
│    │  HF Trainer (no SFTTr.)   │                            │
│    │  • BF16 · LR=1e-4        │                            │
│    │  • max_grad_norm=1.0     │                            │
│    │  • 3 epochs · batch=1    │                            │
│    │  • ~8.5 hours on 4×3060  │                            │
│    └──────────┬───────────────┘                            │
│               │                                            │
│               ▼                                            │
│         Final Model (scale=0.4)                            │
│    (LoRA merged into text decoder)                         │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Merge Quality

The LoRA was tested at multiple merge scales to find the optimal balance between preserving base knowledge and applying the fine-tuned behavior:

Max weight diff measures the largest change in any single weight tensor compared to the base model. Values below 0.02 are considered safe — the model retains base knowledge while incorporating new behaviors. The final merge uses scale=0.4 for a good balance.

Hardware

Hyperparameters

---

✅ What This Model Excels At

• Coding agent tasks — Planning, debugging, building real projects
• Step-by-step reasoning — Autonomous agent trace style
• Tool calling — File operations, shell commands, testing
• Multi-language — Python, C, C++, Go, Java, Rust, Bash, etc.
• Multi-turn — Full trajectory conversations
• Thai + English — Native bilingual capability

---

📊 Dataset Statistics

---

⚠️ Limitations

• 4B model — Smaller than 27B/72B models
• Preview — Early release
• Text-only — Text decoder only (no vision encoder)
• Coding-focused — Excels at coding/debugging, general knowledge may vary

---

💖 Support / โปรดสนับสนุน

If you find this model useful, please consider supporting my work!
หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏

<p align="center"> <img src="https://huggingface.co/hotdogs/Qwen35B-Agent-R2/raw/main/donate.webp" alt="Bitcoin QR — Donate" width="256"> </p>

₿ Bitcoin — BTC:

bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v

Thank you for your support! 🙏✨
ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗

---

🙏 Acknowledgements / ขอบคุณ

• InternScience — For the Agents-A1-4B base model
• greghavens — For the Kimi K3 coding traces dataset
• Qwen Team (Alibaba) — For the Qwen3.5 architecture
• Unsloth AI — For the training optimizations
• All dataset contributors and the open-source AI community ❤️

---

Built with ❤️ by UKA — 18-year-old coder & cybersecurity expert