Alux

Alux

Alux is a fine-tuned version of Qwen2.5-3B-Instruct, converted to GGUF format for fast, efficient inference with llama.cpp and compatible runtimes (Ollama, LM Studio, GPT4All, etc.). It was fine-tuned and quantized using Unsloth, achieving significant training speed-ups.

---

Model Details

---

Available Files

Additional quantization levels (Q5_K_M, Q8_0, etc.) can be added here if you upload more files.

---

Intended Use

Alux is intended for conversational / instruction-following tasks such as:

• General question answering
• Chat assistants
• Text generation and summarization
• Lightweight local deployment on consumer hardware

Out-of-Scope Use

This model should not be used for high-stakes decision-making (medical, legal, financial) without human oversight, and may not perform reliably on languages other than English or highly specialized domains outside its fine-tuning data.

---

How to Use

With llama.cpp

Text-only:

llama-cli -hf bijoy0236/Alux --jinja

Multimodal (if applicable):

llama-mtmd-cli -hf bijoy0236/Alux --jinja

With Ollama

An Ollama Modelfile is included in this repository for easy local deployment:

ollama create alux -f Modelfile
ollama run alux

With llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="bijoy0236/Alux",
    filename="Qwen2.5-3B-Instruct.Q4_K_M.gguf",
)

response = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Hello! What can you do?"}]
)
print(response["choices"][0]["message"]["content"])

---

Training Details

• Base model: Qwen2.5-3B-Instruct
• Method: Supervised fine-tuning (SFT) accelerated with Unsloth
• Hardware/compute: (add details — e.g. GPU type, training time, number of epochs)
• Dataset: (add dataset name/link and a brief description of size and composition)

Fill in the specifics above so users understand what data and process shaped this model's behavior.

---

Limitations and Bias

As with all LLMs, Alux may:

• Produce factually incorrect or outdated information ("hallucinations")
• Reflect biases present in its base model's training data
• Struggle with complex multi-step reasoning or long-context tasks relative to larger models

Users should validate outputs for critical or sensitive applications.

---

---

Citation

If you use this model, please consider citing the base model and Unsloth:

@misc{qwen2.5,
  title={Qwen2.5 Technical Report},
  author={Qwen Team},
  year={2024}
}

---

Contact

For questions or issues, please open a discussion on the model repository.