ThoxNova-12B-Core

ThoxNova-12B-Core — Conservative Chat LoRA

Your AI. Your Data. Your Rules.™

ThoxNova-12B-Core is THOX's conservative chat fine-tune for the THOX stack: a LoRA adapter (rank 8), bundled with Q4_K_M (7.38 GB) and Q5_K_M (8.55 GB) GGUF builds. The same THOX adapter is also published as ThoxNova-12B-Agent with a different persona overlay. Requires a llama.cpp or Ollama build with support for this model family.

Chat template:

<start_of_turn>user
{prompt}<end_of_turn>
<start_of_turn>model

Files

Run it

Bundled GGUF (Ollama)

ollama run Thox-ai/thoxnova-12b-core

Bundled GGUF (llama.cpp)

hf download Thox-ai/ThoxNova-12B-Core --include '*.gguf' --local-dir ./ThoxNova-12B-Core
llama-cli -m ./ThoxNova-12B-Core/thoxnova-12b-core-Q4_K_M.gguf -p "Hello" -n 128

Adapter (Transformers + PEFT)

Load the base model named in this card's metadata, then attach the adapter from this repo:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "<base model named in this card's metadata>"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, "Thox-ai/ThoxNova-12B-Core")

Sized for hosts with at least 16 GB RAM or a GPU with at least 10 GB VRAM.

Use with ThoxOS

ThoxOS loads this model as a host-class assistant through its local model runtime. Serve the bundled GGUF with Ollama on the ThoxOS host:

ollama run Thox-ai/thoxnova-12b-core

ThoxOS points its chat and agent surfaces at the local Ollama endpoint (http://localhost:11434); no cloud account is required and prompts stay on the machine.

Use with MeshStack

On a MeshStack node, pull the model once and serve it to the rest of your mesh:

ollama pull Thox-ai/thoxnova-12b-core
ollama serve

Other devices on the same mesh reach it through the node's OpenAI-compatible endpoint (http://<node>:11434/v1). The adapter form can equally be served from the Hugging Face files with llama.cpp:

hf download Thox-ai/ThoxNova-12B-Core --include '*.gguf' --local-dir ./ThoxNova-12B-Core
llama-cli -m ./ThoxNova-12B-Core/thoxnova-12b-core-Q4_K_M.gguf -p "Hello" -n 128

Intended use

• On-device and edge chat within the THOX stack.
• Not a safety-aligned public assistant unless deployed behind THOX guardrails.
• Out of scope: medical, legal, financial, safety-critical, or other high-stakes decisions without human review; unattended control of physical devices without a confirmation step.

Evaluation

No THOX GATE49 result is published for this model.

Limitations

• No evaluation artifact or training log is published for this model; no quality figure is claimed.
• Models of this size make factual mistakes and can produce confident but wrong output; verify important answers.
• Behaviour and bias reflect the training data, which is mostly English; quality in other languages is lower.
• Quantization trades accuracy for size.

License

Apache-2.0 as declared (see the linked license terms). The THOX adapter and additions in this repo are covered under Apache-2.0; keep the upstream LICENSE/NOTICE with any redistribution.

THOX.ai — Your AI. Your Data. Your Rules.™