Llama-3.2-3B-4bit-classification-routing-gguf

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Thamotharan
• Model type: Embedding
• Language(s) (NLP): English
• **Contact (Mail): **[email protected]

Uses

This model that can be used as a routing agent that will analyze the query and output is given as the single Department Name.

Downstream Use [optional]

Effective for Token reduction in the use of multi agent system by routing the query directly to the particiular department instead of using LLM for reasoning. It is costly for LLM reasoning and have time conplexity issues that is mitigated using this model.

Llama-3.2-3B-4bit-classification-routing-gguf : GGUF

This model was finetuned and converted to GGUF format using Unsloth.

Example usage:

• For text only LLMs: llama-cli -hf Thamo31/Llama-3.2-3B-4bit-classification-routing-gguf --jinja
• For multimodal models: llama-mtmd-cli -hf Thamo31/Llama-3.2-3B-4bit-classification-routing-gguf --jinja

Available Model files:

• Llama-3.2-3B-Instruct.Q4_K_M.gguf

Ollama

An Ollama Modelfile is included for easy deployment. This was trained 2x faster with Unsloth <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>