SmolLM2-135M-Function-Calling

SmolLM2-135M-Function-Calling

Model Description

SmolLM2-135M-Function-Calling is a fine-tuned version of HuggingFaceTB/SmolLM2-135M specifically optimized for function calling tasks. This model has been trained to generate syntactically valid function calls in JSON format, making it suitable for lightweight applications requiring structured function invocation.

Key Achievement: This model achieves 92.18% Structural Validity on BFCL and 97.2% Function Name Accuracy on internal validation, demonstrating strong performance despite its compact size of only 135M parameters.

Attribution

The dataset combination, training strategy, and execution were autonomously achieved by NEO.

Performance Metrics

Use Cases

This model is specifically designed for:

• Edge Device Deployment: Lightweight function calling for resource-constrained environments
• Mobile Applications: Efficient on-device function invocation without cloud dependency
• IoT Systems: Smart device control through structured function calls
• Embedded Systems: Low-latency function execution in embedded applications
• API Gateway Optimization: Fast function routing and parameter extraction

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "gvij/SmolLM2-135M-Function-Calling"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

prompt = """<functions>
[
    {
        "name": "get_weather",
        "description": "Get current weather information",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City name"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
]
</functions>

User: What's the weather in Paris in celsius?

Function Call:"""

inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=150,
        temperature=0.1,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)

Expected output:

{"name": "get_weather", "arguments": {"location": "Paris", "unit": "celsius"}}

Training Details

• Base Model: HuggingFaceTB/SmolLM2-135M
• Training Method: LoRA (Low-Rank Adaptation)
• Function Format: JSON Schema (OpenAI-compatible)
• Training Datasets: Combined function-calling datasets from HuggingFace Hub
• Optimization: Trained for optimal balance between accuracy and structural validity

Model Architecture

• Parameters: 135M
• Architecture: Transformer-based causal language model
• Quantization Support: Compatible with INT8/INT4 quantization for further size reduction
• Context Length: 2048 tokens

Limitations

• Best performance on JSON-formatted function schemas
• May require prompt engineering for optimal results on complex nested function calls
• Performance degrades on extremely long function descriptions (>1000 tokens)

Citation

If you use this model, please cite:

@misc{smollm2-function-calling,
  title={SmolLM2-135M-Function-Calling: Lightweight Function Calling Model},
  author={NEO Agent},
  year={2024},
  publisher={HuggingFace},
  note={Fine-tuned from HuggingFaceTB/SmolLM2-135M}
}

License

Apache 2.0 (inherited from base model)