geolip-aleph-differentiation

geolip-aleph-differentiation

exp_011 — Additive-Conjunctive Differentiation (ACD): can composed micro-alephs be enriched, or does stacking them diverge?

This repository holds the composition-operator study of the GeoLIP aleph program. It is the sibling of geolip-aleph-lm (the single-aleph language-model line) and shares its core: the signed-projective address over K oriented axes on S^(D−1).

The problem

A single aleph (K=64, D=4) is one soft partition — ~7 bits of routing channel, eff-rank ≤ 4. Naively adding more alephs produces cascade noise divergence: free codebooks trained on the same signal fall into the same geometric attractor (redundant partitions), signed disagreement interferes rather than averaging, and accumulated addresses get noisier, not sharper.

The design law

The chain rule of mutual information:

I(Y; A₁,…,A_m) = Σₜ I(Y; Aₜ | A₁..Aₜ₋₁)

Additive differentiation is only additive in information if each stage is conditioned on the previous ones. The four conditioning routes — residual, branch selection, subspace independence, adjudication — define the operator taxonomy under test:

Headline gauge: the marginal-bits curve — estimated I(Y; Aₜ | A₍<t₎) per stage. A structure is enriched iff the curve stays positive as m grows. The estimator is calibrated against an oracle addresser (recovers exact per-level bits on synthetic tasks) and a noise addresser (recovers zero) before any arm is trusted.

The instrument

• notebook - Everything is in here. Everything below Fable put. I don't feel like rewriting it currently. ACD attention however, is quite interesting.

• acd_structures.py — the seven operators behind one interface; address core lifted verbatim from the aleph-lm line; statute gauges (deviation / eff-rank / spread) per stage.

• acd_probe.py — nested globular clusters (Gaussian bubbles with sub-bubbles, ground-truth hierarchy known ⇒ exact per-level information), the marginal-bits estimator with oracle/noise calibration, and composition gauges: cross-stage redundancy, hemisphere cancellation rate, stage SNR.

• acd_forge.py — the automation: JSON arm grammar with hashed identities, a generator that auto-inserts every arm's budget-matched single-aleph twin and the sum divergence control, successive-halving rungs with in-rung kill rules (NaN, gradient blowup, rank collapse), an append-only ledger, and logged verdicts (PROMOTE/PARK/KILL, each with the gauge values that caused it). Results push here under exp011/ each rung.

• acd_lm_adapter.py — Phase 3 (Tier-L): the composed address conditions the byte-trigram AlephLM backbone (α-gated residual at the pre-head seam), so every head predicts through the structure. Includes the Tier-L arm runner, next-byte staged probes, and a synthetic Markov stream for smokes.

• acd_attention.py — Phase 4: composition where information is created. Composed micro-addresses AS the attention feature map (additive kernel over stage addresses; hub math inherited untouched; single mode is parity-gated ≡ stock, Δ=0). phase4_screen() → exp011R/.

Paste order: structures → probe → the aleph-lm cells (geolip-aleph-lm 1–4) → attention → lm_adapter → forge; then phase2_screen() / phase2b_screen() / phase3_screen().

Status

Campaign complete through Phase 5. The findings — including the two laws (aggregation-channel enrichment; the interface law), the captured SUM cascade, the m*=d_in/D saturation constant, and the honest LM-neutrality results across three placements — are written up in ARTICLE.md. Ledgers for every screen live under exp011*/. Open roads: composed-bank apmix (the one untested interface), tree dual-accounting screen, Tier-A scale test.

exp012 — autoregressive differentiation (July 9, 2026)

The sequel campaign lives in exp012_ar/: exp_011 ended on honest LM-neutrality for composed placements; exp012 employs ONE address fully — a byte-LM whose entire next-byte distribution reads from the aleph. Verdicts: the address- bottleneck head beats the unrestricted head 7/7 across all seeds and budgets; the consumption law (slot-parallel reconstructive reads cultivate, hard-tau coefficient heads collapse at any dim); sign-code task-parity (3/3 seeds); the 0.29154 shell as a transit point; the projective-codebook law under pure predictive pressure. Full write-up: exp012_ar/article.md — 48-run ledger, 17 trained specimens, and the complete bed included.

exp013 — augmenting pretrained models (July 10, 2026)

exp013_aug/: the aleph structures applied to FROZEN pretrained models. Headline: GPT-2 124M ppl 38.65 -> 26.53 with aleph relay adapters (1.18M trainable) vs 27.26 for the param-matched MLP ablation — 2/2 seeds, with the gate mechanism visible (aleph gates grow 3x from init, MLP gates shrink below it). Also: the bottleneck prior is substrate-scoped (MLP wins frozen CLIP-L token-AR); the layer law (penultimate is richer but nonlinearly coded); sign-code > soft read in 12/12 pretrained-substrate cells; spelling-AR shows the address extracts more of the surface residue that exists but conjures nothing absent. 18 specimens, all books projective. Full ledger + write-up in exp013_aug/README.md.

exp014 — genetic distillation + memory substrate (July 10, 2026)

exp014_gd/: multi-generational tournaments with the aleph codebook as an explicit heritable genome (GM3 paradigm; Procrustes/GPA consensus). Verdicts: INVERSE EVOLUTION through logit inheritance (KD from near-parity teachers compounds downward — founder-controlled); inheritance pays IFF trunk continuity (organ-only transplants are below-random inits; floor luck beats them); the germline buys STABILITY not score (consensus books hit a lineage fixed point by gen 2); catastrophic parents are NOT absorbed; implants transfer stability, not score. Unifying insight: near-uniform books are near-interchangeable scaffolds — genetic methods pay only where the book's CONTENT is load-bearing. Write-up: exp014_gd/article.md; ledger + 23 champion genomes included.

exp015 — content-bearing heredity (July 10, 2026)

exp015_ch/: exp014's compass executed — tournaments where the prediction channel consumes book IDENTITY (the sign-code head: features are the ±A[win] rows themselves), plus the tree lineage under fair full-weight inheritance. Verdicts: content consumption cultivates LSH fidelity by itself (0.942 → 0.954 with no germline), compressing the germline's score headroom to founder-luck scale; the lineage fixed point replicates on the discrete channel and LOCKS fidelity (<0.001 inter-member spread); the tree inherits under continuity (both lineages monotone; whole-structure fixed point by gen 2); branch revival belongs to continuity — the germline's stationarity freezes routing at zero score cost; and heredity maintains root-routing health that every fresh founder loses. The content gauge separates heredity from lottery far more cleanly than score does. 80-run ledger + 20 champion genomes; write-up in exp015_ch/README.md.

exp016 — heredity judged on a content task (July 11, 2026)

exp016_ct/: the fitness function becomes the variable — training untouched, but evolution SELECTS on a retrieval task the book's discrete code must carry (sign-code Hamming kNN, next-4-byte neighbor agreement; judge validated 0.165 trained vs 0.097 untrained). Verdicts: continuity ascends on the content metric while the consensus germline sits flat, both seeds — the first consistently-signed germline effect, and it's negative (stationarity suppresses the drift content-selection exploits); heredity still beats the lottery (population means: continuity heirs 0.1983 > germline heirs 0.1952 > floor 0.1926); and the headline — catastrophic germline damage is absorbed under trunk continuity (≤0.02 bpb, zero retrieval cost, lineage re-fixes next generation; the same injection cost +0.10 in exp014's organ regime): catastrophe-robustness belongs to the inheritance REGIME, not the operator. Across exp014→016 the honest arc lands: consensus heredity is a robustness mechanism, not a performance mechanism. 112-row ledger + 28 champion genomes; write-up in exp016_ct/README.md. (Same date: exp014/exp015 main brackets replicated at a second tournament seed; exp015's flat-book fixed-point finding amended as seed-dependent — see its README.)

exp017 — prototype aleph constellation (July 11, 2026)

exp017_ac/: the constellation element for the aleph address — the 16s bridge (16 anchors × 16 patches × 3 SLERP strobe phases = the 768 address; aleph D=4 home lifted to the S^15 measurement sphere) as a byte-LM head. Verdicts: the bpb advantage over the certified aleph head was CAPACITY (a param-matched SquaredReLU MLP wins ~0.06 at both seeds — bottlenecks pay when tight); the geometry is the result — the binding constant appears on both layers under pure CE (S^15 anchors drift ~0.287 with 50-81% on the 0.29154 shell; the D=4 codebook beneath lands 0.29-0.33), the CV→0.20 band is native to the S^15 sphere (0.213 at random init) and held by a 1e-3 bank loss at no task cost, and Procrustes calibration proves non-load-bearing for a trainable bank. Value case: the structured 768 address (conditioning/routing/ lookup), not raw bpb. 8-row ledger + 6 checkpoints; write-up in exp017_ac/README.md.

exp018 — experiment 18: exp012 rerun with keystone parameters (July 11, 2026)

exp018_r12/: a labeled rerun (exp012's record untouched) pricing the two keystone avenues the original bed never exercised. Verdicts: the init avenue is task-neutral (geovocab2 regular-pentachoron vertices and a farmed recon-real codebook both land in the certified band; farmed anchors drift less, accelerate nothing); the tied M̂ readout fails in autoregression (+1.0 bpb, codebook starved) — a reconstruction-regime device. Net: exp012's original parameters stand. Write-up: exp018_r12/README.md.

exp019 — the capacity for distilled content retention (July 11, 2026)

exp019_cr/: fact corpora in the byte stream; exact-match recall as the retention gauge; distillation channels vs direct learning across N ∈ {64, 256, 1024}, plus a rule-content generalization block. Verdicts: a capacity cliff between 256 and 1024 facts; the distillation tax is zero-to-negative (teacher logits alone transfer rote content at parity+ and rule content better than ground truth, 2/2 seeds — with a steep clean-bpb cost); no logit leakage without exposure; anchors carry no bytes (a trained codebook from a 256-fact teacher transfers none of them); universal catastrophic forgetting (every channel → 0.000 exact after 1k clean steps) — persistence, not transfer, is the unsolved axis. Rules are learned only fragmentarily and content is format-locked (KD students less so). Write-up: exp019_cr/README.md.

exp020 — generalization: structure vs capacity on a hidden rule (July 11, 2026)

exp020_gen/: the exp019 rule task raced across six structural arms with an exact param-matched control. Verdicts: no structure lifts rule induction off the ~0.26 plateau (exact 0.000 in all 12 cells); the aleph bottleneck is the best generalizer (+25% over its matched free head, both seeds); the inverse law — generalization ordering reverses modeling strength (the constellation head models best and generalizes worst; memorization ease substitutes for rule induction); format diversity covers seen formats at full rule level but not novel ones. The measured memorization↔generalization axis doubles as a component registry for slider/registry-style composites. Write-up: exp020_gen/README.md.

exp021 — persistent memory via addressed external storage (July 11, 2026)

exp021_ps/: a key-value store outside the gradient path, keyed by the model's own discrete addresses (sign-code / constellation patch-winner), written once after training — persistence by construction IF the addresses survive interference. They don't: addresses drift with the trunk representation, not the address parameters (keys scramble 72–91%; freezing the entire key path changes nothing — the decomposition arm). Cheap replay is insufficient; coarse codes drift less but unusably. The constructive requirement: durable addressed memory needs a write-time-frozen key encoder — the memory keyer and the learning trunk must be separate systems. Write-up: exp021_ps/README.md.

Status: this line is ON HOLD (July 11, 2026)

exp011→exp021 complete. The byte-LM differentiation series closes here as a finished measurement program: the consumption law, the bottleneck's scope, the projective-codebook law, the KD regime map (weak→strong works; parity inverts), heredity-as-robustness, the inverse law (memorization ease suppresses rule induction), and the frozen-keyer requirement. The program continues on pretrained generative substrates at geolip-aleph-qwen, which composes the certified components (relay adapters, sign-code interface, frozen-keyer registry, weak→strong distillation) into a utilizable generation structure. Experiment numbering restarts at 1 there.

Reproducibility

Every experiment package (exp012_ar/, exp013_aug/, exp014_gd/, exp015_ch/) is standalone: it carries its own copies of the shared harness (geolip_vitals.py, ar_differentiation_bed.py, read_codebook.py) and a repro.py loader — python repro.py smokes on CPU, flags forward to verdict runs on GPU. Data roots default to ./data (override with GEOLIP_DATA). Each README's snippet has been verified by execution; the exp013 track-b1 snippet reproduces its published table exactly from a fresh cache.

Relation to prior work

Residual-expert quantization (RQ-MoE, SwitchCodec/REVQ), hierarchical conditional routing (S'MoRE), and learned latent cluster trees (TreeVAE lineage) each hold one piece. Unoccupied: signed antipodal addresses, statute-governed stage geometry, prediction flowing through the composed structure, and marginal information per stage as the design criterion. That conjunction is this repository.

License

MIT.