Downloads · 30 days
0
sourav-paramanya/qwen2.5-3b-esim-assistant-GGUF
qwen2.5-3b-esim-assistant-GGUF is a machine learning model from sourav-paramanya. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This repository contains an optimized 4-bit GGUF (q4km) instance of a fine-tuned Qwen2.5-3B-Instruct model, specialized for autonomous telecom workflows and precise eSIM package recommendations.
Downloads · 30 days
0
Access
Public
Updated Jun 17, 2026
Parameters
3.1B
6.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors6.2 GB · 100%
From the Hugging Face model README
This repository contains an optimized 4-bit GGUF (q4_k_m) instance of a fine-tuned Qwen2.5-3B-Instruct model, specialized for autonomous telecom workflows and precise eSIM package recommendations.
The model was trained via Unsloth using QLoRA parameters and compiled natively through llama.cpp hooks to enable ultra-low latency inference on local commodity CPUs and budget cloud servers.
https://esim.oss.net.bd/checkout/...) without formatting degradation.q4_k_m - Recommended balanced medium quantization)To execute this compiled model locally or interface it with your backend FastAPI microservices using Ollama, implement the following setup:
.gguf file from the Files section.Modelfile and input the following block:FROM ./qwen2.5-3b-esim-assistant-Q4_K_M.gguf
# Define ChatML Template Boundaries
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .User }}<|im_start|>user
{{ .User }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Output }}<|im_end|>"""
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"