LocoOperator-4B-GGUF

Quantized at Romarchive

<div align="center"> <img src="assets/loco_operator.png" width="55%" alt="LocoOperator" /> </div> <br> <div align="center">

</div>

Introduction

LocoOperator-4B is a 4B-parameter tool-calling agent model trained via knowledge distillation from Qwen3-Coder-Next inference traces. It specializes in multi-turn codebase exploration — reading files, searching code, and navigating project structures within a Claude Code-style agent loop. Designed as a local sub agent, it runs via llama.cpp at zero API cost.

Key Features

• Tool-Calling Agent: Generates structured <tool_call> JSON for Read, Grep, Glob, Bash, Write, Edit, and Task (subagent delegation)
• 100% JSON Validity: Every tool call is valid JSON with all required arguments — outperforming the teacher model (87.6%)
• Local Deployment: GGUF quantized, runs on Mac Studio via llama.cpp at zero API cost
• Lightweight Explorer: 4B parameters, optimized for fast codebase search and navigation
• Multi-Turn: Handles conversation depths of 3–33 messages with consistent tool-calling behavior

Performance

Evaluated on 65 multi-turn conversation samples from diverse open-source projects (scipy, fastapi, arrow, attrs, gevent, gunicorn, etc.), with labels generated by Qwen3-Coder-Next.

Core Metrics

The model perfectly learned when to use tools vs. when to respond with text (100% presence alignment). Tool type mismatches are between semantically similar tools (e.g. Grep vs Read) — different but often valid strategies.

Tool Distribution Comparison

<div align="center"> <img src="assets/tool_distribution.png" width="80%" alt="Tool Distribution Comparison" /> </div>

JSON & Argument Syntax Correctness

LocoOperator-4B achieves perfect structured output. The teacher model has 11 tool calls with missing required arguments (empty arguments: {}).

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "LocoreMind/LocoOperator-4B"

# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

# prepare the messages
messages = [
    {
        "role": "system",
        "content": "You are a read-only codebase search specialist.\n\nCRITICAL CONSTRAINTS:\n1. STRICTLY READ-ONLY: You cannot create, edit, delete, move files, or run any state-changing commands. Use tools/bash ONLY for reading (e.g., ls, find, cat, grep).\n2. EFFICIENCY: Spawn multiple parallel tool calls for faster searching.\n3. OUTPUT RULES: \n   - ALWAYS use absolute file paths.\n   - STRICTLY NO EMOJIS in your response.\n   - Output your final report directly. Do not use colons before tool calls.\n\nENV: Working directory is /Users/developer/workspace/code-analyzer (macOS, zsh)."
    },
    {
        "role": "user",
        "content": "Analyze the Black codebase at `/Users/developer/workspace/code-analyzer/projects/black`.\nFind and explain:\n1. How Black discovers config files.\n2. The exact search order for config files.\n3. Supported config file formats.\n4. Where this configuration discovery logic lives in the codebase.\n\nReturn a comprehensive answer with relevant code snippets and absolute file paths."
    }
]

# prepare the model input
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512,
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()

content = tokenizer.decode(output_ids, skip_special_tokens=True)
print(content)

Local Deployment

For GGUF quantized deployment with llama.cpp, hybrid proxy routing, and batch analysis pipelines, refer to our GitHub repository.

Training Details

Known Limitations

• First-tool-type match is 65.6% — the model sometimes picks a different (but not necessarily wrong) tool than the teacher
• Tends to under-generate parallel tool calls compared to the teacher (76 vs 89 total calls across 65 samples)
• Preference for Bash over Read may indicate the model defaults to shell commands where file reads would be more appropriate
• Evaluated on 65 samples only; larger-scale evaluation needed

License

MIT

Acknowledgments

• Qwen Team for the Qwen3-4B-Instruct-2507 base model
• MS-SWIFT for the training framework
• llama.cpp for efficient local inference
• Anthropic for the Claude Code agent loop design that inspired this work