Downloads · 30 days
21
7% of all-time downloads
Orchestrator-8B-GGUF
gannima/Orchestrator-8B-GGUF
Orchestrator-8B-GGUF is a machine learning model from gannima. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as other.
This repository contains GGUF quantized versions of nvidia/Orchestrator-8B, optimized for local inference.
All-time downloads
282
Public
Repo size
20.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf20.5 GB · 100%
At a glance
- Library
- gguf
- License
- other
- Access
- Public
- Created
- Dec 1, 2025
- Updated
- Dec 1, 2025
- SHA
- 9a80384f
Base models
Model card
From the Hugging Face model README
gannima/Orchestrator-8B-GGUF
This repository contains GGUF quantized versions of nvidia/Orchestrator-8B, optimized for local inference.
License
This model is governed by the NVIDIA Open Model License.
Available Quantizations
| Filename | Quant Type | Size | Use Case |
|---|---|---|---|
nvidia-Orchestrator-8B-q8_0.gguf | q8_0.gguf | 8.2G | High Accuracy |
nvidia-Orchestrator-8B-q6_k.gguf | q6_k.gguf | 6.3G | Balanced High |
nvidia-Orchestrator-8B-q4_k_m.gguf | q4_k_m.gguf | 4.7G | Fast / Low VRAM |
Usage with llama.cpp
1. Install
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make -j
2. Run Server
Example for Q6_K model:
./llama-server \
-m nvidia-Orchestrator-8B-q6_k.gguf \
-c 32768 \
-ngl 99 \
--port 8080 \
--chat-template chatml \
--temp 0.6 --top-p 0.9 --min-p 0.05
- Library
- gguf
- License
- other
- Created
- Dec 1, 2025
- Updated
- Dec 1, 2025