ocular-qwen1.7b-mixed01

ocular-qwen1.7b-mixed01 (F16 GGUF)

A DoRA finetune of Qwen3-1.7B on a personal corpus, merged into the base weights and exported to GGUF at full precision. Private.

F16 is the archival artifact, not the fast one. It is unquantized, so it is memory-bandwidth bound exactly like bf16 — measured 16 tok/s on an RTX 4050 where the bf16 transformers version did ~57. Its purpose is to be the lossless source for llama-quantize, and a checkpoint that does not decay.

What it was trained on

22.64M tokens, 25,700 records:

soup is six months of the author's own conversations with Claude across two dialects (claude.ai export + Claude Code JSONL), with the agentic layer — tool calls and results — deliberately kept rather than stripped. It is roughly 59% tool traffic by bytes.

graphsoup is that corpus walked through a symbolic link graph (JANUS), producing sequences of real chunks in orders that never occurred. Real material, novel adjacency, no model authorship.

Method

base        Qwen/Qwen3-1.7B
adapter     DoRA, rank 32, alpha 64
targets     q,k,v,o,gate,up,down projections
trained     bf16, seq 2048, 1 epoch, 1,404 steps
final       train_loss 2.142
merged      into base, exported F16 GGUF (310 tensors, 3.45 GB)

Behaviour

Measured against the untuned base on a hand-rolled bench:

The headline is the third row. The tuned model answers wrong about a third as often, and spends the difference reaching for a tool rather than guessing. Base: "I cannot run a shape audit without access." Tuned: emits a Bash call.

The 45% Rosetta share is visible and probably too high. Asked "what is 17 times 23" it writes a C# program rather than answering 391. Asked to run a shape audit it writes Go. If you retrain, turn Rosetta down first.

Known defects

1. No record separator. Training records were concatenated with nothing marking the boundaries, so block edges taught transitions between unrelated documents as continuations. Conversations got partial cover from <|im_end|>; nothing else did.
2. Packing slices mid-record every 2048 tokens. 98.8% of records fit whole, so this hits the long tail — which is where the longest sessions live, including one at 187k tokens.
3. weight_decay = 0.0. Never set.

Regurgitation

Probed with three corpus-specific prefixes. No verbatim training data returned — only format and register ([THINKING] markers, connective-tissue phrases). Three probes is not an audit. Kept private accordingly.

Provenance

Trained 2026-08-21 on a rented RTX 5090 (vast.ai), ~96 min, roughly $1.30. Merged and converted on the laptop. Full trail in the source repo: runs/2026-08-21/ and KNOWN-DEFECTS.md.