LumiAgent-9B-v2-GGUF

LumiAgent-9B-v2-GGUF

LumiAgent-9B-v2 is a coding and agent-oriented fine-tune derived from empero-ai/Qwythos-9B-v2.

This repository contains text-only GGUF exports intended for llama.cpp, LM Studio, Goose and compatible agent runtimes.

Available quantizations

Training

• Base model: empero-ai/Qwythos-9B-v2
• Training precision: BF16 LoRA
• LoRA rank: 64
• LoRA alpha: 64
• Training context: 4096 tokens
• Training examples: 5,900
• Holdout examples: 100
• Epochs: 1
• Optimizer steps: 738
• Mean training loss: 0.4715
• Trainable parameters: approximately 116.4 million
• Final export: text-only
• Multimodal projector: not included
• MTP auxiliary head: excluded

The training mixture emphasized:

• coding and debugging;
• tool calling and unfamiliar tool schemas;
• MCP, ACP, plugins, extensions and reusable skills;
• planning, execution and result verification;
• agent loops, retries and rollback;
• web research and authorized scraping workflows;
• subagent delegation;
• Romanian and English instructions.

The training dataset itself is not included in this repository.

Local Arc B580 observations

The following are local observations, not standardized benchmark results.

Q5_K_M

• Intel Arc B580 12 GB
• LM Studio / llama.cpp runtime
• Generation speed: approximately 38.47 tokens/second
• Prompt processing: approximately 447 tokens/second
• Loaded context in the tested configuration: 49,408 tokens

Q8_0

• Intel Arc B580 12 GB
• Generation speed: approximately 19.39 tokens/second
• Loaded context in the tested configuration: 17,664 tokens

Q5_K_M is recommended for long agent sessions and tool-heavy workflows.

Important limitation

LumiAgent should not be treated as proof that generated code is correct.

In direct chat testing, the model produced plausible-looking Python code that still contained syntax and testing errors. For serious coding work, run it through an agent environment that can:

1. create and inspect files;
2. execute commands;
3. run tests;
4. read actual errors;
5. repair the implementation;
6. verify the final result.

Recommended runtimes include Goose with the Developer extension, or another agent controller with terminal access and mandatory verification.

Suggested settings

Q5_K_M

• GPU offload: maximum
• Flash Attention: enabled
• KV cache on GPU: enabled
• Context: start with 16K or 32K
• Temperature for coding: 0.2 to 0.5

Q8_0

• GPU offload: maximum
• Context: 8K to 16K
• Use primarily for quality comparisons or shorter tasks

Safety and authorization

Security-related functionality should only be used on systems you own, authorized environments, local laboratories and permitted CTF challenges.

Integrity

SHA-256 hashes are provided in SHA256SUMS.txt.

License and attribution

LumiAgent-9B-v2 is derived from empero-ai/Qwythos-9B-v2, distributed under the Apache License 2.0.

Repository: Laurentiupad/LumiAgent-9B-v2-GGUF