ThoxLLM-327M-v2

ThoxLLM-327M-v2

Your AI. Your Data. Your Rules.

Production from-scratch 327M decoder with standard 2-projection SwiGLU (GGUF-clean).

What this is

• Trained from scratch (THOX-owned) — Apache-2.0.
• Standard SwiGLU (matched-FLOPs, ffn 3413) makes GGUF export clean via K-quants.
• config.json has no architectures/model_type key, so it is not vanilla-AutoModel loadable — load via the THOX thoxllm package or use the GGUF repo.
• Dataset provenance note: parent card says THOX-internal corpus; the GGUF card says Pile-uncopyrighted — reconcile before publishing a data statement.

Architecture (from config)

Intended use

On-device / edge text generation within the THOX stack. Not a safety-aligned public assistant unless deployed behind THOX guardrails.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Thox-ai/ThoxLLM-327M-v2")
model = AutoModelForCausalLM.from_pretrained("Thox-ai/ThoxLLM-327M-v2")

Note: this checkpoint uses the custom THOX thoxllm decoder; config.json has no architectures key, so vanilla AutoModel loading may not work — use the THOX thoxllm package or the GGUF build.

Links

• GGUF: Thox-ai/ThoxLLM-327M-v2-GGUF
• Docs: https://docs.thox.ai

---

THOX.ai LLC — Your AI. Your Data. Your Rules. · On-device and private by design.

Temporary demo Space: tommytracx/ThoxLLM-327M-v2-Space (org Space paused on quota).

<!-- THOX-STANDARD:BEGIN -->

Files

Use with ThoxOS

This repo has no GGUF. For ThoxOS / llama.cpp / Ollama / LM Studio use Thox-ai/ThoxLLM-327M-v2-GGUF:

hf download Thox-ai/ThoxLLM-327M-v2-GGUF --include "*.gguf" --local-dir ./ThoxLLM-327M-v2-GGUF
ollama run hf.co/Thox-ai/ThoxLLM-327M-v2-GGUF:Q4_K_M

Use with MeshStack

Serve the GGUF from Thox-ai/ThoxLLM-327M-v2-GGUF with llama-server --host 0.0.0.0 --port 8080 (OpenAI-compatible /v1) or Ollama (:11434) and register that endpoint with your MeshStack coordinator.

Device fit (by file size only)

• model.safetensors (648.34 MB): Medium (<2.5 GB): devices with ≥4 GB RAM — recent phones, laptops, Apple Silicon Macs, MeshStack edge hosts.

RAM needed ≈ file size + context (KV cache) + runtime overhead. These tiers are derived from file size alone; no throughput, TOPS or battery figures are claimed here.

Licence and attribution

Trained from scratch by THOX.ai LLC; Apache-2.0.

Out of scope

• Medical, legal, financial, safety-critical or other high-stakes decisions without human review.
• Unattended control of physical devices or actuators without a confirmation step.
• Any use prohibited by the upstream licence or acceptable-use policy.

Limitations and bias

• Small and quantized models make factual mistakes and can produce confident but wrong output; verify important answers.
• Behaviour and bias reflect the training data, which is mostly English web/instruction text; quality in other languages is lower unless stated above.
• Quantization trades accuracy for size; lower-bit files degrade more.

---

THOX.ai LLC — Your AI. Your Data. Your Rules.™ · thox.ai · Contact: [email protected]

<!-- THOX-STANDARD:END -->