ghostai-alpha

GhostAI Alpha

Alpha SFT of Qwen3.5-0.8B for Ghost AI's on-device planner/executor: it takes a natural-language crypto request plus a memory block and emits a structured plan JSON, then compiles each plan step into an exact tool-call JSON.

Published for in-app integration testing only. See Limitations — this is not a beta candidate and its eval numbers are not meaningful.

Files

GGUF (llama.cpp / on-device)

Transformers format

MTP head

The MTP head is shipped separately and is not part of model.safetensors or any GGUF. See mtp/README.md before using it — pairing an untrained draft head with a fine-tuned target model degrades speculative-decoding acceptance rates.

Model

• Architecture: Qwen3_5ForCausalLM, 24 layers, hidden 1024, GQA (8 Q / 2 KV heads), vocab 248,320, 262K context. Gated DeltaNet hybrid — layer_types alternates three linear_attention layers to one full_attention.
• Parameters: 752.4M. The base checkpoint is multimodal (Qwen3_5ForConditionalGeneration); the 100.6M vision tower is dropped for text-only use.
• MTP head removed. The base checkpoint's Multi-Token-Prediction head (15 tensors) is not carried through, and mtp_num_hidden_layers is set to 0 so llama.cpp does not expect a 25th block. Speculative decoding via MTP is therefore unavailable.

Training

DeepSpeed ZeRO-3, bf16, 2× A100-40GB.

Usage

llama-cli -m ghostai-alpha-Q4_K_M.gguf -ngl 99 --temp 0 --jinja \
  --chat-template-kwargs '{"enable_thinking":false}' \
  -sys "You are Ghost AI, an on-device mobile crypto assistant. Produce a structured plan JSON matching the v1 schema." \
  -p "swap 2 SOL to USDC"

Roughly 254 tok/s generation for Q4_K_M on a single A100.

Production use assumes grammar-constrained decoding — the plan schema when planning, the individual tool's JSON schema when executing. Do not rely on unconstrained output being schema-valid.

Limitations

Do not read the eval metrics above as generalization. They are reported for reproducibility only.

1. Train/eval contamination. The evaluation split was drawn from the same scenario corpus as training; 25% of eval rows are byte-identical to a training row. Train and eval loss track each other almost exactly (0.2014 vs 0.202 at epoch 1), which is the contamination, not generalization.
2. Templated targets. Training data was generated deterministically, not distilled from a teacher. Across 400 plans there are only 33 unique summary strings, and across 72 clarification plans only 3 unique questions (one of which is the contentless "Could you clarify?"). The model reproduces these templates verbatim — expect "Referenced token(s): ..." and similar strings to appear as memorized output.
3. No reasoning traces. Assistant turns are bare plan JSON, so the model has no trained short-reasoning behavior.
4. Partial tool coverage. Only 25 of the 36 manifest tools appear in any execution trace. The web_search / web_research / web_fetch family is entirely absent, so retrieval-grounded behavior is untrained.
5. Refusal/risk behavior is unvalidated. Risk and refusal categories are ~30% of the data mix but their targets are templated, so declines should not be trusted without independent evaluation.

A teacher-distilled corpus with a genuinely held-out eval set is required before any tier model is a shipping candidate.

License

Base model Qwen/Qwen3.5-0.8B is Apache 2.0; that license governs this derivative.