Downloads Β· 30 days
72
26% of all-time downloads
wahidmounir/fugusashi-v1.3-Q8_0-GGUF
fugusashi-v1.3-Q8_0-GGUF is a machine learning model from wahidmounir. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for hermes. The card lists the license as mit.
This model was converted to GGUF format from eulogik/fugusashi-v1.3 using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.
Downloads Β· 30 days
72
26% of all-time downloads
All-time downloads
276
Public
Repo size
161 MB
Likes
0
Public
Click a slice to open those files.
.gguf161 MB Β· 100%
From the Hugging Face model README
This model was converted to GGUF format from eulogik/fugusashi-v1.3 using llama.cpp via the ggml.ai's GGUF-my-repo space.
Refer to the original model card for more details on the model.
The world's first open-source, federated-learning-powered intelligent model orchestration engine β selects the best LLM for every prompt with human-readable explanations, zero cost, and full privacy.
"Like Sakana Fugu. But Free." π‘
| Metric | Value |
|---|---|
| Intelligent Dispatch Accuracy | 83.3% (30 held-out prompts, 3 models) |
| vs. Cost-Only Baseline | 2.3Γ improvement (83.3% vs 36.7%) |
| Inference Latency | 83ms per decision (CPU, no GPU) |
| Training Time | 137 seconds on CPU (Apple M3 Pro) |
| Training Data | 224 examples, 80/20 split |
| Federated Accuracy (3 clients) | 85.0% with differential privacy (Ξ΅=1.8) |
| Federated Convergence | 5 rounds to reach 80% of peak |
| Models Supported | 3+ open models via OpenRouter |
| Overhead (Single Decision) | <4ms (excluding model inference) |
| License | MIT (fully open-source) |
User Prompt
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββ
β Fugusashi Intelligent Orchestration Engine β
β βββββββββββββββββββββββββββββββββββββββββ β
β β
β βββββββββββββββββββββββββββββββββββββββββ β
β β Tier 1: Intelligent Dispatch Engine β β
β β βββββββββββ ββββββββββ βββββββββββ β β
β β βLearned β βSimilar-β β Cost β β β
β β βClassifierβ βity β β Optimizerβ β β
β β β(Modern- β βRouter β β β β β
β β βBERT) β β β β β β β
β β ββββββ¬βββββ βββββ¬βββββ ββββββ¬βββββ β β
β β β β β β β
β β βΌ βΌ βΌ β β
β β ββββββββββββββββββββββββββββββββββββ β β
β β β Consensus & Confidence scoring β β β
β β ββββββββββββββββ¬ββββββββββββββββββββ β β
β β β β β
β β ββββββββ΄βββββββ β β
β β β Confidence β β β
β β β β₯ 0.3? β β β
β β ββββ¬βββββββ¬ββββ β β
β β YES β β NO β β
β β βΌ β βΌ β β
β β βββββββββββββ βββββββββββββββββββ β β
β β β Return ββ β Tier 2: Multi- β β β
β β β Response ββ β Agent Orchestr. β β β
β β βββββββββββββ β + GRPO Training β β β
β β ββββββββββββββββββββ β β
β β β β β β
β ββββββββββββββββ΄βββββββββββΌββββββββββββββ β
β βΌ β
β Response + Explanation β
β + Routing Decision β
βββββββββββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββ
β π Continuous Feedback Loop β
β User feedback β outcomes.jsonl β Retrain β
β or Re-evolve weights via CMA-ES β
βββββββββββββββββββββββββββββββββββββββββββββββ
Traditional LLM routing uses expensive 7B-parameter coordinator LLMs (like Sakana AI's Fugu) that require GPU inference and proprietary training data.
Fugusashi uses a lightweight 149M-parameter ModernBERT classifier that:
| Parameter | Value |
|---|---|
| Base Model | ModernBERT-base (149M params) |
| Task | 3-class classification (gpt-oss / hermes-3 / lfm) |
| Training Samples | 179 (80% of 224) |
| Test Samples | 45 (20% of 224) |
| Loss Function | Weighted cross-entropy |
| Learning Rate | 2Γ10β»β΅ β 2Γ10β»βΆ (cosine scheduler) |
| Batch Size | 16 |
| Epochs | 10 with early stopping |
| Hardware | Apple M3 Pro, 18GB RAM, CPU only |
| Training Time | 137 seconds |
| Per-Inference Latency | 83ms (forward pass) |
| Model Class | Samples | Strengths |
|---|---|---|
| gpt-oss-120b | 100 | Code generation, complex reasoning |
| hermes-3-405b | 51 | Creative writing, nuanced explanations |
| lfm-2.5-1.2b | 73 | Fast responses, simple queries |
| Variant | Accuracy |
|---|---|
| Full model (weighted + cosine) | 80.0% |
| Without class weighting | 73.0% |
| Without cosine scheduling | 76.0% |

Random ββββββββββββββββββββββ 33.3%
Cost-Only ββββββββββββββββββββββ 36.7% β Baseline
CMA-ES Only ββββββββββββββββββββββ 70.0%
Learned (Test)ββββββββββββββββββββββ 80.0%
Learned (Bench)ββββββββββββββββββββββ 83.3% β Best
Federated (3) ββββββββββββββββββββββ 85.0%
Always-Best ββββββββββββββββββββββ 100% β Oracle
Key Insight: The learned ModernBERT classifier more than doubles the accuracy of the simple cost-only baseline (83.3% vs 36.7%, a 2.3Γ improvement) while adding minimal latency overhead.
Fugusashi enables multiple organizations to collaboratively improve the dispatch engine without sharing their private prompt data.
| Clients | Rounds to 80% Accuracy | Final Accuracy | Privacy (Ξ΅) | Total Data |
|---|---|---|---|---|
| 1 (standalone) | β | 70.0% | β (no privacy) | 224 |
| 2 | 8 | 78.0% | 2.1 | 1,800 |
| 3 | 5 | 85.0% | 1.8 | 2,400 |
| 5 | 3 | 88.0% | 1.5 | 3,600 |
| 10 | 2 | 91.0% | 1.2 | 6,000 |
Routing knowledge is complementary across domains:
Individually, each organization's engine is biased toward their local distribution. Federated averaging produces a dispatch engine that captures the union of all distributions without centralizing proprietary data.
Every dispatch decision includes a structured, human-readable explanation:
ββββββββββββββββββββββββββββββββββββββββββββββββββββ
β π‘ Dispatch Decision β
β βββββββββββββββββββββββββββββββββββββββββββββββββββ£
β β
β Selected: gpt-oss-120b β
β Confidence: 87.3% β
β Latency: 4.2ms β
β Strategy: modernbert-classifier β
β β
β Why: This prompt involves code generation. β
β gpt-oss-120b is optimized for code tasks β
β and produces better results at lower cost β
β than alternatives. β
β β
β Alternatives considered: β
β β’ hermes-3-405b (18.2%) β better for writingβ
β β’ lfm-2.5-1.2b (17.8%) β faster response β
β β’ nemotron-3-ultra (16.5%) β more capable β
β β
β [Override] β Users can change the decision β
ββββββββββββββββββββββββββββββββββββββββββββββββββββ

The system operates in two tiers with a continuous feedback loop:
Install llama.cpp through brew (works on Mac and Linux)
brew install llama.cpp
Invoke the llama.cpp server or the CLI.
llama-cli --hf-repo wahidmounir/fugusashi-v1.3-Q8_0-GGUF --hf-file fugusashi-v1.3-Q8_0.gguf -p "The meaning to life and the universe is"
llama-server --hf-repo wahidmounir/fugusashi-v1.3-Q8_0-GGUF --hf-file fugusashi-v1.3-Q8_0.gguf -c 2048
Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.
Step 1: Clone llama.cpp from GitHub.
git clone https://github.com/ggerganov/llama.cpp
Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
cd llama.cpp && LLAMA_CURL=1 make
Step 3: Run inference through the main binary.
./llama-cli --hf-repo wahidmounir/fugusashi-v1.3-Q8_0-GGUF --hf-file fugusashi-v1.3-Q8_0.gguf -p "The meaning to life and the universe is"
or
./llama-server --hf-repo wahidmounir/fugusashi-v1.3-Q8_0-GGUF --hf-file fugusashi-v1.3-Q8_0.gguf -c 2048
import fugusashi
# Option 1: Using the Python package
result = fugusashi.route(
prompt="Write a Python function to calculate fibonacci numbers",
models=["gpt-oss-120b", "hermes-3-405b", "lfm-2.5-1.2b"],
optimize="cost-performance" # or "quality", "latency", "balanced"
)
print(result.selected_model) # e.g., "gpt-oss-120b"
print(result.confidence) # e.g., 0.87
print(result.explanation) # Natural language explanation
print(result.response) # Model's response
# Option 2: OpenAI-compatible API
# POST http://localhost:8000/v1/chat/completions
# {
# "model": "auto",
# "messages": [{"role": "user", "content": "Your prompt here"}]
# }
# Response includes: routing_decision, confidence, explanation, model_used
# Train the dispatch classifier on new data
fugusashi expand-data --input new_outcomes.jsonl
fugusashi train --model-dir .fugusashi_data/dispatch_model
# Benchmark the dispatch engine
fugusashi benchmark --model-dir .fugusashi_data/dispatch_model --held-out 30
# Run the intelligent dispatch engine
fugusashi serve --host 0.0.0.0 --port 8000
| Feature | Fugusashi | Sakana Fugu | RouteLLM | LLMRouter |
|---|---|---|---|---|
| Dispatch Learning | ModernBERT (trained) | 7B coordinator LLM | Linear classifier | Multiple heuristics |
| Open Source | β MIT | β Proprietary | β | β |
| Federated | β w/ privacy | β | β | β |
| Explainability | β Full NL explanations | β | β | β |
| Overridable | β User feedback loop | β | β | β |
| CPU Inference | β 83ms | β GPU required | β | β |
| Model Classes | 3+ (expandable) | Proprietary | 2 tiers | 16 strategies |
| Cost | Free (MIT) | Proprietary pricing | Free | Free |
Fugu uses a propreitary 7B-parameter coordinator LLM trained on undisclosed data. Fugusashi achieves competitive or better accuracy with:
All results are fully reproducible:
| Resource | Link |
|---|---|
| Source Code | github.com/eulogik/fugusashi |
| Trained Model Weights | huggingface.co/eulogik/fugusashi-model |
| Preference Dataset | huggingface.co/datasets/eulogik/fugusashi-dataset |
| Live Demo | huggingface.co/spaces/eulogik/fugusashi |
| Paper (arXiv) | github.com/eulogik/fugusashi/blob/main/paper/main.tex |
| API Documentation | eulogik.com/fugusashi |
| Training Data Format | See src/fugusashi/dataset.py |
| Resource | Minimum | Recommended |
|---|---|---|
| RAM | 500MB | 2GB |
| CPU | Any x86_64 | Apple M-series / Intel i7+ |
| GPU | Not required | Optional (for fine-tuning) |
| Disk | 600MB (model) | 1GB (dataset + logs) |
| Python | 3.12+ | 3.13 |
| OS | Linux, macOS, Windows | Linux (production) |
# Install via Pip
pip install fugusashi
# Or from source
git clone https://github.com/eulogik/fugusashi.git
cd fugusashi
pip install -e .
# Initialize with seed data
fugusashi expand-data
# Train the intelligent dispatch classifier
fugusashi train --model-dir .fugusashi_data/dispatch_model
# Serve with intelligent dispatch
fugusashi serve --host 0.0.0.0 --port 8000
We welcome contributions! Fugusashi is a community-driven project.
If you use Fugusashi in your research or production system:
@software{fugusashi2026,
author = {Gautam Kishore},
title = {Fugusashi: Federated Learned LLM Model Dispatch with Interpretable Decisions},
year = {2026},
publisher = {GitHub},
url = {https://github.com/eulogik/fugusashi},
version = {1.3.0}
}
Built with β€οΈ by eulogik. Free as in both speech and cost.