Qwen2.5-7B-SFT-Capybara

Qwen2.5-7B-SFT-Capybara

This model is a fine-tuned version of Qwen/Qwen2.5-7B trained on the trl-lib/Capybara dataset using SFT with LoRA adapters.

Overview

Qwen2.5-7B-SFT-Capybara is a language model optimized using SFT. Supervised Fine-Tuning (SFT) trains the model to follow instructions by learning from high-quality demonstration data.

Key Features

• High-Quality Fine-Tuning: Trained on N/A carefully curated examples
• Efficient Training: Uses LoRA (Low-Rank Adaptation) with 4-bit quantization
• Strong Performance: Achieves N/A token accuracy on evaluation set
• Optimized for Inference: Available in multiple formats including GGUF quantizations

Model Details

Training Information

Training Configuration

LoRA Configuration

Training Metrics

Dataset

This model was trained on the trl-lib/Capybara dataset.

Usage

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "ermiaazarkhalili/Qwen2.5-7B-SFT-Capybara"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the sum of 2 + 2?"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)

Using Pipeline

from transformers import pipeline

generator = pipeline("text-generation", model="ermiaazarkhalili/Qwen2.5-7B-SFT-Capybara", device_map="auto")
messages = [{"role": "user", "content": "Explain the concept of machine learning."}]
output = generator(messages, max_new_tokens=256, return_full_text=False)
print(output[0]["generated_text"])

4-bit Quantized Inference

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4"
)

model = AutoModelForCausalLM.from_pretrained(
    "ermiaazarkhalili/Qwen2.5-7B-SFT-Capybara",
    quantization_config=quantization_config,
    device_map="auto"
)

GGUF Versions

For CPU or mixed CPU/GPU inference, GGUF quantized versions are available at: ermiaazarkhalili/Qwen2.5-7B-SFT-Capybara-GGUF

Using with Ollama

ollama pull hf.co/ermiaazarkhalili/Qwen2.5-7B-SFT-Capybara-GGUF:Q4_K_M
ollama run hf.co/ermiaazarkhalili/Qwen2.5-7B-SFT-Capybara-GGUF:Q4_K_M "Hello!"

Limitations

• Language: Primarily trained on English data
• Knowledge Cutoff: Limited to base model's training data cutoff
• Hallucinations: May generate plausible-sounding but incorrect information
• Context Length: Fine-tuned with 2,048 token limit
• Safety: Not extensively safety-tuned; use with appropriate guardrails

Intended Use

Recommended Uses

• Research on language model fine-tuning
• Educational purposes
• Personal projects
• Prototyping conversational AI

Out-of-Scope Uses

• Production systems without additional safety measures
• Medical, legal, or financial advice
• Generating harmful or misleading content

Training Framework

• TRL: 0.24.0
• Transformers: 4.57.3
• PyTorch: 2.9.0
• Datasets: 4.3.0
• PEFT: 0.18.0
• BitsAndBytes: 0.49.0

Citation

@misc{ermiaazarkhalili_qwen2.5_7b_sft_capybara,
    author = {ermiaazarkhalili},
    title = {Qwen2.5-7B-SFT-Capybara: Fine-tuned Qwen2.5-7B on Capybara},
    year = {2026},
    publisher = {Hugging Face},
    howpublished = {\url{https://huggingface.co/ermiaazarkhalili/Qwen2.5-7B-SFT-Capybara}}
}

Acknowledgments

• Base model developers at Qwen
• Hugging Face TRL Team for the training library
• Dataset creators and contributors
• Compute Canada / DRAC for HPC resources

Contact

For questions or issues, please open an issue on the model repository.