sift-1b-sft

Sift-1B-SFT (Supervised Fine-Tuned Adapter)

<!-- Quick Summary -->

Sift-1B-SFT is a 1.5B-parameter Small Language Model adapter fine-tuned via 4-bit QLoRA on top of Qwen/Qwen2.5-1.5B-Instruct. It serves as Stage 1 in the Sift-1B model suite, explicitly trained to transform raw natural language queries into deterministic, type-safe JSON function calls and multi-agent routing decisions.

• Developer: Sanatan Singh
• Model Type: PEFT / QLoRA Adapter (Supervised Fine-Tuning)
• Base Model: Qwen/Qwen2.5-1.5B-Instruct
• Language(s): English (en)
• License: MIT
• Repository: GitHub — SanatanSinghVishen/Sift-1B
• DPO Aligned Model: SanatanSinghVishen/sift-1b-dpo
• GGUF Quantized: SanatanSinghVishen/sift-1b-gguf

---

Model Details

Overview

General-purpose Large Language Models (LLMs) often suffer from latency, token cost, and conversational "fluff" when used merely to route user intents or extract structured parameters. Sift-1B-SFT addresses this by providing a lightweight, low-latency model optimized specifically for:

1. Deterministic Function Calling: Extracting structured JSON arguments adhering to strict schemas.
2. Multi-Agent Intent Routing: Identifying user intent and generating standard route tags in sub-50ms inference windows.
3. Local Edge Deployment: Operating efficiently within 4 GB VRAM budgets (RTX 3050, Apple Silicon, or CPU edge hardware).

---

Intended Uses

Direct Use

• Structured Data Extraction: Converting unstructured user prompts into ChatML tool calls.
• Local API Middleware: Acting as an intent classification and parameter extraction backend for local-first applications.

Out-of-Scope Use

• General conversational chat, creative writing, or open-ended Q&A.
• Complex multi-step mathematical reasoning without tool assistance.

---

How to Get Started with the Model

You can load and run Sift-1B-SFT using Hugging Face transformers and peft:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_name = "Qwen/Qwen2.5-1.5B-Instruct"
adapter_name = "SanatanSinghVishen/sift-1b-sft"

# 1. Load Tokenizer & Base Model
tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# 2. Load SFT LoRA Adapter
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()

# 3. Format Input Query with ChatML
messages = [
    {
        "role": "system", 
        "content": "You are a function calling agent. Output only valid JSON tool calls."
    },
    {
        "role": "user", 
        "content": "Schedule a team sync with Alex tomorrow at 10:00 AM."
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

# 4. Generate Structured JSON Response
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.1)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)