ThoxLLM-327M-v2-GGUF

<div align="center"> <img src="hero-family.png" alt="The THOX four-device family" width="100%"> <h1>ThoxLLM-327M-v2-GGUF</h1>

GGUF Builds of the From-Scratch THOX 327M — k-quant GGUF builds of Thox-ai/ThoxLLM-327M-v2, THOX's from-scratch 327M language model, for llama.cpp, Ollama, and LM Studio

GGUF: thoxllm-327m-v2.f16.gguf, thoxllm-327m-v2.q2_k.gguf, thoxllm-327m-v2.q3_k_m.gguf, thoxllm-327m-v2.q4_k_m.gguf, thoxllm-327m-v2.q5_k_m.gguf

</div>

Your AI. Your Data. Your Rules.™

Status: PUBLISHED — GGUF BUILDS. K-quant GGUF builds (F16, Q2_K, Q3_K_M, Q4_K_M, Q5_K_M) of Thox-ai/ThoxLLM-327M-v2, THOX's from-scratch 327M language model, for llama.cpp, Ollama, and LM Studio. The model card of record is the main repository's. See Files for exactly what this repo contains.

Summary

K-quant GGUF builds of Thox-ai/ThoxLLM-327M-v2, THOX's from-scratch 327M language model, for llama.cpp, Ollama, and LM Studio. Only K-quants are published (Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, F16): the model's intermediate size is not divisible by 32, so legacy Q4_0 / Q8_0 builds are not available.

Your AI. Your Data. Your Rules.™

Highlights

• K-quant GGUF builds of Thox-ai/ThoxLLM-327M-v2, THOX's from-scratch 327M language model, for llama.cpp, Ollama, and LM Studio.
• bash hf download Thox-ai/ThoxLLM-327M-v2-GGUF --include '*.gguf' --local-dir ./ThoxLLM-327M-v2-GGUF llama-cli -m ./ThoxLLM-327M-v2-GGUF/thoxllm-327m-v2.q4_k_m.gguf -p "Hello" -n 128

Specifications

Files

Run it

hf download Thox-ai/ThoxLLM-327M-v2-GGUF --include '*.gguf' --local-dir ./ThoxLLM-327M-v2-GGUF
llama-cli -m ./ThoxLLM-327M-v2-GGUF/thoxllm-327m-v2.q4_k_m.gguf -p "Hello" -n 128

ollama run hf.co/Thox-ai/ThoxLLM-327M-v2-GGUF:Q4_K_M

This is a base (non-instruct) model: use plain completion prompts, not chat.

Platforms

Intended use

• On-device and edge text generation within the THOX stack.
• Not a safety-aligned public assistant unless deployed behind THOX guardrails.
• Out of scope: medical, legal, financial, safety-critical, or other high-stakes decisions without human review; unattended control of physical devices without a confirmation step.

Training data and procedure

Trained by THOX from scratch as part of the THOX small-model research line. The corpus and procedure are as described in this card at release; no external base model was used.

Evaluation

No THOX GATE49 result is published for this model.

Limitations

• A 327M base model: limited reasoning and factual reliability; verify important answers.
• Lower-bit quantizations degrade more; Q2_K is a size experiment, not a quality build.
• Behaviour and bias reflect the training data, which is mostly English; quality in other languages is lower.

License

Apache-2.0. Quantized from a THOX model trained from scratch by THOX.ai LLC; covered under Apache-2.0.

THOX.ai — Your AI. Your Data. Your Rules.™

Provenance

Published by THOX.ai as Thox-ai/ThoxLLM-327M-v2-GGUF. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.

Citation

@misc{thoxai2026thoxllm327mv2gguf,
      title={ThoxLLM-327M-v2-GGUF},
      author={THOX.ai},
      year={2026},
      url={https://huggingface.co/Thox-ai/ThoxLLM-327M-v2-GGUF}
}