Downloads · 30 days
18
100% of all-time downloads
Keitsuna123/llama-3.2-1b-fc-sft-full
llama-3.2-1b-fc-sft-full is a machine learning model from Keitsuna123. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as llama3.2.
Code & writeup: https://github.com/keitake123/llama-function-calling-study
Downloads · 30 days
18
100% of all-time downloads
All-time downloads
18
Public
Repo size
62.3 MB
Likes
0
Public
Click a slice to open those files.
.safetensors45.1 MB · 72%
From the Hugging Face model README
Code & writeup: https://github.com/keitake123/llama-function-calling-study
Supervised fine-tune of meta-llama/Llama-3.2-1B-Instruct for function calling, trained on the Salesforce xLAM dataset. This is the v1 SFT model in a four-model study comparing SFT, data scaling, GRPO, and supervised refusal training for small-model function calling.
Evaluated on the Berkeley Function Calling Leaderboard v4, compared to the base model:
| Category | Base | This model (v1 SFT) |
|---|---|---|
| simple_python | 75.0 | 77.5 |
| multiple | 50.5 | 74.0 |
| live_simple | 31.8 | 57.0 |
| live_multiple | 7.3 | 38.8 |
| live_relevance | 43.8 | 93.8 |
| live_irrelevance | 67.3 | 16.9 |
| irrelevance | 35.8 | 5.8 |
| parallel | 44.0 | 1.0 |
| parallel_multiple | 15.0 | 2.0 |
Large gains on call-required categories (+23–50 pts). Note the irrelevance regression (35.8→5.8): training on single-call xLAM data — which contains no "no-call" examples — teaches the model to always call a function. This regression is addressed in the companion Path B model.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained("Keitsuna123/llama-3.2-1b-fc-sft-full")
model = AutoModelForCausalLM.from_pretrained(
"Keitsuna123/llama-3.2-1b-fc-sft-full", torch_dtype=torch.bfloat16, device_map="auto"
)
messages = [{"role": "user", "content": "What's the weather in Tokyo?"}]
tools = [{"type": "function", "function": {
"name": "get_weather", "description": "Get the weather for a location",
"parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}
}}]
text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(out[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True))
# {"name": "get_weather", "parameters": {"location": "Tokyo"}}
| Model | Description |
|---|---|
| fc-sft-full | v1 SFT on xLAM (this model) |
| fc-sft-v2-merged | SFT on xLAM + distilabel (data-scaling ablation) |
| fc-grpo | GRPO for irrelevance recovery (negative result) |
| fc-pathb | Supervised refusal training (effective irrelevance fix) |
This work builds on:
@misc{taketsuna2026_fc_smallmodel,
title = {Small-Model Function Calling: Comparing SFT, Data Scaling, GRPO, and Supervised Refusal Training},
author = {Taketsuna, Keiichi},
year = {2026},
howpublished = {\url{https://huggingface.co/Keitsuna123}},
note = {Llama-3.2-1B function-calling post-training study on BFCL v4}
}