qwen-capybara-medium-gguf

qwen-capybara-medium-gguf

This is a GGUF conversion of evalstate/qwen-capybara-medium, which is a LoRA fine-tuned version of Qwen/Qwen2.5-0.5B.

Model Details

• Base Model: Qwen/Qwen2.5-0.5B
• Fine-tuned Model: evalstate/qwen-capybara-medium
• Training: Supervised Fine-Tuning (SFT) with TRL
• Format: GGUF (for llama.cpp, Ollama, LM Studio, etc.)

Available Quantizations

Usage

With llama.cpp

# Download model
huggingface-cli download evalstate/qwen-capybara-medium-gguf qwen-capybara-medium-q4_k_m.gguf

# Run with llama.cpp
./llama-cli -m qwen-capybara-medium-q4_k_m.gguf -p "Your prompt here"

With Ollama

1. Create a Modelfile:

FROM ./qwen-capybara-medium-q4_k_m.gguf

1. Create the model:

ollama create qwen-capybara -f Modelfile
ollama run qwen-capybara

With LM Studio

1. Download the .gguf file
2. Import into LM Studio
3. Start chatting!

Training Details

This model was fine-tuned using:

• Dataset: trl-lib/Capybara (1,000 examples)
• Method: Supervised Fine-Tuning with LoRA
• Epochs: 3
• LoRA rank: 16
• Hardware: A10G Large GPU

License

Inherits the license from the base model: Qwen/Qwen2.5-0.5B

Citation

@misc{qwen-capybara-medium-gguf,
  author = {evalstate},
  title = {Qwen Capybara Medium GGUF},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/evalstate/qwen-capybara-medium-gguf}
}

---

Converted to GGUF format using llama.cpp