MiniCPM5-1B-Agentic-DPO-LoRA

MiniCPM5-1B-Agentic-LoRA

Better Finetune Repo V2 CLICK HERE

TL;DR: This is a 1-Billion parameter autonomous agent model fine-tuned for Python dictionary tool execution. It was fine-tuned using QLoRA on a single 16GB T4 GPU using the Salesforce xLAM function-calling dataset. Note: This model does NOT use a standard chat template; it uses a custom raw string format with <user>, <tools>, and <calls> tags.

This is a fine-tuned version of the MiniCPM5-1B model, specifically optimized for agentic tool use and function calling.

This model was trained on a Google Colab T4 GPU using Supervised Fine-Tuning (SFT) with QLoRA on the Salesforce/xlam-function-calling-60k dataset to master tool usage.

🌱 Potential for Improvement: Due to the strict 3-hour GPU quota limits on Google Colab, this model was trained on a randomly sampled subset of only 10,000 examples from the full 60k xLAM dataset. Consequently, this model has significant room for improvement! Re-training for multiple epochs on the full 60,000-sample dataset would likely yield a dramatic increase in tool-calling precision.

The repository includes the merged PyTorch weights as well as highly optimized GGUF (FP16, Q8_0, Q4_K_M) files for fast CPU/Edge inference.

🤝 Credits & Acknowledgements

This model is built on top of MiniCPM5-1B, developed by the OpenBMB team. We extend our immense gratitude to the original creators for releasing their powerful, highly efficient model under the Apache 2.0 license, which made this fine-tuning and agentic research project possible.

🚀 Model Details

• Base Model: openbmb/MiniCPM5-1B
• Training Dataset: Salesforce/xlam-function-calling-60k (10k subset used due to GPU limits)
• Training Method: SFT + QLoRA
• Hardware Used: NVIDIA T4 (Google Colab, 12.7GB System RAM)
• Primary Use Case: API calling and tool execution.

📝 Prompt Format (CRITICAL)

DO NOT use tokenizer.apply_chat_template() with this model. This model was trained on raw strings with a very specific, custom XML-like schema format for structuring the input.

CRITICAL WARNING ABOUT JSON & NEWLINES:

1. The model outputs Python dictionary literals (which use single quotes '), NOT valid JSON (which strictly requires double quotes "). You must use ast.literal_eval() instead of json.loads() to parse the output!
2. Do NOT put newlines inside the tags. The dictionaries must start immediately after the tags.
3. The tool definition schema MUST match the xLAM dataset schema (nested parameters dict).
4. The output function calls will use the key arguments, not parameters.

You must format your prompt EXACTLY like this (including the newlines):

<user>Where can I find live giveaways for beta access and games?</user>

<tools>{'name': 'live_giveaways_by_type', 'description': 'Retrieve live giveaways based on the specified type.', 'parameters': {'type': {'description': 'The type of giveaways to retrieve (e.g., game, loot, beta).', 'type': 'str', 'default': 'game'}}}</tools>

<calls>

The model will then generate the corresponding tool calls inside the <calls> block as Python dictionary literals.

---

💻 Usage (Hugging Face transformers)

import torch
import ast
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "ewinregirgojr/MiniCPM5-1B-Agentic-DPO-LoRA"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Because the LoRA adapters were merged permanently into the base model weights, 
# you can load this repository directly as a standalone CausalLM without needing the peft library!
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto", 
    trust_remote_code=True
)

# You MUST construct the prompt as a raw string! Notice the lack of newlines inside <tools>.
# And you MUST use the xLAM nested-parameter schema for your tools!
prompt = """<user>Where can I find live giveaways for beta access and games?</user>

<tools>{'name': 'live_giveaways_by_type', 'description': 'Retrieve live giveaways based on the specified type.', 'parameters': {'type': {'description': 'The type of giveaways to retrieve (e.g., game, loot, beta).', 'type': 'str', 'default': 'game'}}}</tools>

<calls>"""

# If you are on a CPU, remove `.to("cuda")` below
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

# CRITICAL: You must pass eos_token_id to generate() so it knows when to stop!
outputs = model.generate(
    **inputs, 
    max_new_tokens=250, 
    temperature=0.1,
    eos_token_id=tokenizer.eos_token_id
)

# Decode the output
response_text = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response_text.strip())

# To parse the output into a Python dict, DO NOT use json.loads()! Use ast.literal_eval()
# parsed_tool = ast.literal_eval(response_text.strip())

---

🏎️ Usage (Llama.cpp Python via GGUF)

For ultra-fast inference on edge devices or CPUs, use the provided Q8 or Q4 GGUF files.

# Install with GPU support (if applicable)
# CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python huggingface_hub
import ast

from llama_cpp import Llama
from huggingface_hub import hf_hub_download

# Download the Q8 Quantized Model
model_path = hf_hub_download(
    repo_id="ewinregirgojr/MiniCPM5-1B-Agentic-DPO-LoRA", 
    filename="minicpm5-agentic-Q8_0.gguf"
)

# Load Model
llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_gpu_layers=-1, # Set to 0 if you are running strictly on CPU
    verbose=False
)

prompt = """<user>Where can I find live giveaways for beta access and games?</user>

<tools>{'name': 'live_giveaways_by_type', 'description': 'Retrieve live giveaways based on the specified type.', 'parameters': {'type': {'description': 'The type of giveaways to retrieve (e.g., game, loot, beta).', 'type': 'str', 'default': 'game'}}}</tools>

<calls>"""

# Generate
response = llm(
    prompt,
    max_tokens=512,
    temperature=0.1,
    echo=False,
    stop=["}"] # <--- CRITICAL: Forces the model to stop looping!
)

response_text = response['choices'][0]['text'].strip()
# Append the closing brace since the stop string is excluded from the output
response_text += "}"
print(response_text)

# parsed_tool = ast.literal_eval(response_text)

❓ Frequently Asked Questions (FAQ) for Developers

Q: Can I run this model on a pure CPU without a GPU?
A: Yes! The best way to run this model on a CPU is by downloading the GGUF files (Q4_K_M or Q8_0) and using llama.cpp. In the Python code above, simply set n_gpu_layers=0 to force all computation onto your CPU. If you are using the transformers library instead, make sure you don't call .to("cuda") on your inputs.

Q: Because the LoRA was merged into the model, does that mean I can't fine-tune it anymore?
A: No, you can absolutely still fine-tune it! When a LoRA is merged, its weights are mathematically added into the base model's weights permanently. This means this model (ewinregirgojr/MiniCPM5-1B-Agentic-DPO-LoRA) can now be treated as a brand new base model. If you want to improve it (for example, by training it on the remaining 50,000 xLAM samples), you can simply load this model as your base, attach a brand new LoRA adapter to it, train it, and merge it again.

Q: Can I use tokenizer.apply_chat_template() with this model?
A: No. This model was fine-tuned using a custom raw string structure (<user>, <tools>, <calls>). Using the standard Hugging Face chat template will format the prompt incorrectly and break the tool-calling behavior.

Q: Why do I get a JSONDecodeError when I use json.loads() on the output?
A: Because of the way the training data was concatenated in Python, the model outputs Python dictionary strings (which use single quotes '), rather than valid JSON (which requires double quotes "). You must use ast.literal_eval(output) to parse the generated tool calls.

Q: Should I use skip_special_tokens=True or False?
A: You should use skip_special_tokens=True. The <user>, <tools>, and <calls> tags were treated as normal text during training and were never added to the tokenizer's special tokens list. Therefore, it is completely safe to skip special tokens—this will remove the [EOS] token at the end of the generation without deleting your tool tags.

Q: How do I format the <tools> and <calls> tags?
A: The dictionaries must start immediately after the opening tags and end immediately before the closing tags, with no newlines in between. For example: <tools>{'name': 'my_tool'}</tools>.

🛠️ Build Pipeline Details

For developers wishing to replicate this build on severely RAM-constrained environments like Google Colab T4 (12.7GB RAM):

1. Model Loading: The LoRA weights were merged permanently into the base model on the CPU using device_map="cpu" and safe_merge=True to prevent VRAM and system RAM fragmentation.
2. GGUF Conversion: Hugging Face weights were converted to FP16 GGUF directly on disk.
3. Quantization: llama.cpp was compiled using cmake --target llama-quantize to strictly build the quantization binary in under 60 seconds, bypassing the full project build to prevent OS-level out-of-memory (cc1plus killed) errors.