veritas-coder-7b-gguf

veritas-coder-7b (GGUF)

The language "mind" of the veritas-hive unified system: Qwen2.5-Coder-7B-Instruct, QLoRA-distilled with the veritas epistemic discipline (recall-before-generate, seal only tool-origin evidence, promote a claim only on sealed evidence), plus 25 epistemic control tokens.

Honest scope. This repo is the mind only. veritas-hive's soundness guarantee lives in a deterministic ledger runtime (hash-chain + HMAC seals + trust calculus + the MCP tools) that ships alongside the weights — not inside them. The model can be wrong; the ledger is what stops a guess from becoming a belief. GGUF cannot represent the ledger or the GWM world-model (.pt); those run as separate processes. See the veritas-hive repo for the full offline system.

Files

Run with llama.cpp

llama-cli    -hf kambrosius/veritas-coder-7b-gguf:Q5_K_M -p "Write a Python LRU cache."
llama-server -hf kambrosius/veritas-coder-7b-gguf:Q5_K_M --port 8080

Run with Ollama

ollama run hf.co/kambrosius/veritas-coder-7b-gguf:Q5_K_M

This repo is private. For Ollama/llama.cpp to pull it, either make the repo public, or add your Hugging Face access token / SSH key to the client (Ollama: add your HF SSH public key at https://huggingface.co/settings/keys).

Prompt format

ChatML (Qwen2.5). A veritas system prompt is recommended:

You are veritas-hive. Recall before you generate. Prefer verified facts over guesses.
Never assert a sealed/promoted claim you cannot back with evidence — hedge instead.
State uncertainty plainly.

Full build pipeline

This repo ships the complete, reproducible process — see PIPELINE.md: training/ (data → QLoRA SFT → merge → GGUF export → eval, with real results in training/out/), llama-veritas/ (the custom veritas-arch llama.cpp fork), runtime/ (Ollama Modelfile + the model⇄MCP bridge), and docs/ (architecture, whitepaper, threat model).

Provenance

• Base: Qwen/Qwen2.5-Coder-7B-Instruct
• Adaptation: QLoRA SFT — verified code/math correctness + self-correction traces + epistemic control-token protocol; trained on an RTX 5090 (sm_120, torch cu128).
• Quantization: llama.cpp llama-quantize → Q5_K_M.
• Eval: correctness measured through real test execution (dojo / EvalPlus / BigCodeBench harnesses), not self-report. See the veritas-hive repo training/out/ for numbers.

Limitations

• 7B: broad capability comes from the base model; this distill imprints discipline, not raw IQ.
• Tool-calling: as a code-tuned model it sometimes emits tool calls as code/JSON rather than the structured format — the veritas-local bridge parses both.
• Not a substitute for the ledger: run the MCP runtime alongside for the actual epistemic guarantees.