ThoxNano-360M

ThoxNano-360M

THOX-branded on-device model for the ThoxClip tier of the THOX device program, fine-tuned from HuggingFaceTB/SmolLM-360M on the THOX factory chat + agent/tool-use corpus.

Status (2026-10-28)

• Trained weights are published in this repo. The edge-wave fine-tune completed on 2026-10-27: HF Jobs relaunch 6ae15d7a41e90535df76ddfa (namespace Thox-ai) finished COMPLETED and uploaded the merged weights on 2026-10-28T02:30Z (repo commits cc59bb2 "SFT on Thox-ai/thox-factory-sft-v5 (LoRA r32, 3.0 epochs, merged)" and d3b4cad "Training summary"). training_summary.json in this repo records the run: 205 rows, 3 epochs, final training_loss 3.124.
• First attempt failed before training. Job 6ae14e1f41e90535df76dd37ff8 errored during tokenization because the SmolLM base tokenizer ships no chat template. The trainer in Thox-ai/thox-edge-sft was fixed with a ChatML fallback (commit c250f9bb1b8e20b71f5c38c73ccfbcdac6a0303e) and relaunched; the relaunch produced the weights now in this repo.
• The GGUF in this repo is now the fine-tune. ThoxNano-360M.Q4_K_M.gguf was regenerated from the trained safetensors in the finishing step on 2026-10-28 (llama.cpp conversion + Q4_K_M quantization on TX083) and replaced the 2026-08-11 base repack. SHA-256 in the Files table.
• Factuality gate: could not complete (2026-10-28). The thoxllm-factory factuality gate was run against the trained GGUF twice; both runs aborted part-way because the model fell into a repetition loop on a gate question (Ollama "token repeat limit reached"). No gate score exists for this model. Treat its factual reliability as unproven and, given the repetition failure, weak; deployed surfaces must supply facts via serving-side grounding. Broader benchmark evaluation is still pending.
• Smoke test (2026-10-28, TX083, Ollama): asked "What are you?", the model echoed its THOX device-assistant system text for ThoxClip (with some garbled tokens) rather than answering conversationally — the identity skin is present but the model leaks template text. Recorded as-is.
• Before this wave, the repo carried the 2026-10-23 verified repack (weights differing byte-for-byte from the upstream base, no training log). Those weights were replaced by the trained weights above.

Intended use

On-device assistant / tool-routing for the THOX device program (ThoxClip tier): short conversational replies, THOX house voice, and routing intents to device and mesh functions, fully offline.

Out of scope

Not for safety-critical, medical, or legal decisions without a human in the loop. Not a general-purpose capability upgrade over the base model; the fine-tune is a THOX behavior and voice skin (see Limitations).

Training data and procedure

• Data: Thox-ai/thox-factory-sft-v5 train.jsonl, 205 rows of THOX chat + agent/tool-use examples, staged for the wave in the dataset repo Thox-ai/thox-edge-sft.
• Method: LoRA SFT with TRL (train_edge_sft.py in Thox-ai/thox-edge-sft), LoRA rank 32, learning rate 1e-4, 3 epochs, adapter merged into the base weights. Final training_loss 3.124 (see training_summary.json).
• Chat template: the base SmolLM tokenizer ships no chat template, so the trainer applies a ChatML fallback; the resulting template ships in this repo as chat_template.jinja.
• Compute: Hugging Face Jobs under the Thox-ai namespace (first attempt 6ae14e1f41e90535df76dd37ff8, failed on tokenization; relaunch 6ae15d7a41e90535df76ddfa, COMPLETED).

Evaluation

• Factuality gate (2026-10-28): ERROR, no score. Two runs against the trained Q4_K_M GGUF both aborted on a model repetition loop ("token repeat limit reached") before completing the 49-question set. The gate neither passed nor produced a fail score; the repetition failure itself is recorded as a negative quality signal.
• Identity smoke test: the model echoes its THOX/ThoxClip system text (see Status) — identity present, conversational quality weak.
• Pending: held-out benchmarks, perplexity/capability comparisons vs the base model, and on-device (ThoxClip hardware) validation. Nothing in this card should be read as a quality claim.

Files

Use with ThoxOS

As of the 2026-10-28 finishing step, the GGUF below is the trained fine-tune (see Status).

Download the file once and run it fully offline on a ThoxOS host (or any Linux/macOS/Windows machine):

pip install -U "huggingface_hub[cli]"
hf download Thox-ai/ThoxNano-360M ThoxNano-360M.Q4_K_M.gguf --local-dir ./ThoxNano-360M
llama-cli -m ./ThoxNano-360M/ThoxNano-360M.Q4_K_M.gguf -p "Hello" -n 128

Ollama (pulls straight from the Hub):

ollama run hf.co/Thox-ai/ThoxNano-360M:Q4_K_M

A same-named entry also exists in the Ollama library: ollama.com/Thox-ai/thoxnano-360m (contents not re-verified in this pass; prefer the hf.co/ pull above for a pinned file).

LM Studio: search Thox-ai/ThoxNano-360M in the model browser (or lms get Thox-ai/ThoxNano-360M), pick ThoxNano-360M.Q4_K_M.gguf, and load it.

Use with MeshStack

Serve the model as an OpenAI-compatible endpoint on any MeshStack node, then register that endpoint with your MeshStack coordinator (protocol openai, base URL http://<node-ip>:8080/v1; or protocol ollama at http://<node-ip>:11434 if you serve it with Ollama).

llama-server -m ./ThoxNano-360M/ThoxNano-360M.Q4_K_M.gguf --host 0.0.0.0 --port 8080 -c 4096
curl http://<node-ip>:8080/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Hello"}]}'

Device fit (by file size only)

• ThoxNano-360M.Q4_K_M.gguf (270.59 MB): Small (<600 MB): single-board computers with ≥1 GB RAM, phones, any laptop/desktop.
• model.safetensors (723.67 MB, F16): host-side loading and conversion; quantize before deploying to small devices.

RAM needed ≈ file size + context (KV cache) + runtime overhead. These tiers are derived from file size alone; no throughput, TOPS or battery figures are claimed here.

Licence and attribution

This model is Apache-2.0. Upstream HuggingFaceTB/SmolLM-360M is Apache-2.0 (© Hugging Face); the licence carries forward.

Limitations and bias

• Small and quantized models make factual mistakes and can produce confident but wrong output; verify important answers.
• Behaviour and bias reflect the training data, which is mostly English web/instruction text; quality in other languages is lower unless stated above.
• Quantization trades accuracy for size; lower-bit files degrade more.
• The fine-tune corpus is small (205 rows). It teaches THOX identity, house voice and tool-routing behavior; it does not add new general capability. The only evaluation attempted so far is the factuality gate, which the model could not complete due to repetition loops (see Evaluation).

Provenance

• Base model: HuggingFaceTB/SmolLM-360M (apache-2.0).
• Training data: Thox-ai/thox-factory-sft-v5 (205-row train split), staged via Thox-ai/thox-edge-sft.
• Trainer fix: Thox-ai/thox-edge-sft commit c250f9b (ChatML fallback for the missing base chat template).
• Jobs (HF Jobs, Thox-ai namespace): first attempt 6ae14e1f41e90535df76dd37ff8 (failed, tokenization); relaunch 6ae15d7a41e90535df76ddfa (COMPLETED 2026-10-27, weights uploaded 2026-10-28T02:30Z).
• Weight upload commits: cc59bb2 and d3b4cad3237af627592d6a8355f9b11c40873fdf in this repo.

---

THOX.ai LLC — Your AI. Your Data. Your Rules.™ · thox.ai · Contact: [email protected]

Wave B retrain (2026-10-28)

Retrained on the Wave B corpus (Thox-ai/thox-factory-sft-v5, file train_wave_b.jsonl, 982 rows, SHA-256 6665a8cbd79197c61a9db8119b743faedc3e469407364bb553ec9478e90a5784): the Wave A corpus expanded with campaign-oracle, device-fact, tool-routing, support, and factuality-gate-style rows; 12 v5 rows asserting unpublished chipset identities were excluded. Same recipe as Wave A (LoRA r32 alpha 64 merged, LR 1e-4 cosine, warmup_steps 0.05, 3 epochs, max_length 1024), run locally on KnightHub (RTX 4060 Ti, CUDA). Final training_loss 1.5150 (Wave A: 3.124, on the 205-row corpus; losses are not comparable across corpora).

Factuality gate (49 questions, weights only, no retrieval): Wave A FAIL, ERROR (gate aborted on a repetition loop; no score) (TX083) -> Wave B FAIL, 23/49, score 2.959/5 (KnightHub; TX083 was unreachable during the Wave B finish, see the fleet record wave-b-20261028/RUN.md). The verdict remains FAIL under the gate policy, which allows zero hard failures (Wave B: 11 hard failures). The score improved on Wave A, so this Wave B artifact replaces the Wave A artifact as latest under the track's publish rule. Factual answers from the bare weights remain unreliable in the weak categories; deploy with serving-side grounding for facts.

GGUF updated: ThoxNano-360M.Q4_K_M.gguf, 270,590,336 B, SHA-256 b1c6dcc9c91a8664c420edcc9dbd9f47967f37b5abe193a2a4f5a37ecd904876 (manual llama.cpp pipeline: convert_hf_to_gguf f16 -> llama-quantize Q4_K_M, KnightHub). Ollama thox-ai/thoxnano-360m:q4_k_m and :latest carry the same GGUF.