geolip-aleph-qwen-3.5-0.8b-instruct

geolip-aleph-qwen-3.5-0.8b-instruct

Second-model validation of the Pq234](https://huggingface.co/p Pq234) program on Qwen/Qwen3.5-0.8B — a dual-tower VLM whose LLM tower is hybrid: 18 Gated-DeltaNet linear-attention blocks + 6 full-attention blocks (indices 3,7,11,15,19,23), d=1024, 248,320-token vocab, tied embeddings. Every law certified on the Qwen2.5-0.5B line is re-tested here under one discipline: preregistered rules where applicable, strong zero-shot baselines (this is a post-trained model), single-seed results labeled as candidates, honest negatives shipped as findings.

The dispatch under test is the aleph closed-form addressing M̂ = Σₖ sinh(uₖ)Aₖ / Σₖ cosh(uₖ) — dense signed mixing, frozen key path, no selectors, no top-k, no load-balancing losses.

Substrate (frozen at the gate)

fp32 throughout (TF32 off), gradient checkpointing + chunked CE (248K-vocab logits never materialized per-sequence), memory fraction 0.92 on a 48GB card. Gates passed before any science: template constants discovered (not assumed) — <|im_end|> / <tool_call> identical to the 2.5 line; control vocab bands (30000–33063) collision-free against the 248K tokenizer; adapter wrap fires on both DeltaNet and full-attention blocks; fp32 DeltaNet fallback path benched at 3548 tok/s prefill; a 9k-token training step fits in 16.22GB under the riders. See substrate/v35_substrate.py.

Experiments

Zero-shot baselines (Night 0: 15 vision tasks × 200 images, bf16, greedy)

The instruct model's fingerprint before any adapter — the baseline every conditioning claim is read against. schema_valid = parseable + schema-conformant JSON; score = task-specific deterministic judge.

Laws under test (running tally)

Port surfaces (what did NOT transfer from 2.5)

1. Tool-call render format: Qwen3.5's template emits XML-parameter blocks (<function=…><parameter=…>), not <tool_call>{json}</tool_call>. The 2.5-line parser silently zeroes every valid output (see exp002's amendment).
2. Batch-1 decode is a ~32 tok/s ceiling on the hybrid trunk with the naive torch DeltaNet fallback (one CPU core pinned, GPU idle). Batched generation recovers 10–13×; all judges in this series batch.
3. Fused linear-attention kernels (fla) are three-way blocked on torch 2.4.1 (fla 0.5.x needs triton ≥3.3; torch 2.4 pins 3.0; fla 0.1–0.2 needs torch ≥2.5 APIs). This series runs the torch fallback uniformly — slower, but numerically consistent across every run.

All runs on a single RTX 6000 Ada (48GB), wall-clock ledgered.