telegram-bot-phi35mini-nlu-merged

Telegram Bot NLU - Fine-tuned Phi-3.5-mini

Fine-tuned Phi-3.5-mini-instruct for intent classification and entity extraction in a Telegram bot.

Model Details

• Base Model: microsoft/Phi-3.5-mini-instruct (3.8B parameters)
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Training Data: 170 examples across 6 intents
• Training Time: 5 minutes on RTX 3090 Ti
• Final Loss: 10.4

Intents

• check_weather
• monitor_chess
• goodbye
• help
• show_menu
• cancel

Entities

• city
• url
• tournament_id
• player_snr
• round
• tournament

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "CyberSecify/telegram-bot-phi35mini-nlu-merged",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "CyberSecify/telegram-bot-phi35mini-nlu-merged",
    trust_remote_code=True
)

# Format prompt
system_msg = "You are an NLU assistant. Classify intents and extract entities. Return JSON."
user_msg = "what is the weather in london?"
prompt = f"<|system|>{system_msg}<|end|><|user|>{user_msg}<|end|><|assistant|>"

# Generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.1)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Training Details

• Dataset: 170 examples (Weather: 120, Chess: 12, Simple: 38)
• Epochs: 3
• Batch Size: 4 (effective 16 with gradient accumulation)
• Learning Rate: 2e-4
• LoRA Config: r=16, alpha=32, dropout=0.05
• Trainable Parameters: 3.1M (0.16% of total)