Downloads · 30 days
35
4% of all-time downloads
dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-verlet-instability
semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-verlet-instability is a text generation model from dimitarpg13. Use it when you need the model to write or continue text. It is set up for pytorch. The card lists the license as cc-by-4.0.
This repository holds the two full-scale training attempts ( \\(\gamma = 0.10\\) and \\(\gamma = 0.30\\)) that followed the d=384 gamma sweep, together with a complete SCAF causal-leak and numerical-stiffness audit of…
Downloads · 30 days
35
4% of all-time downloads
All-time downloads
859
Public
Repo size
8.3 GB
Likes
0
Public
Click a slice to open those files.
.pt8.3 GB · 100%
From the Hugging Face model README
This repository holds the two full-scale training attempts ( \(\gamma = 0.10\) and \(\gamma = 0.30\)) that followed the d=384 gamma sweep, together with a complete SCAF causal-leak and numerical-stiffness audit of both. Neither run is a successfully completed model. Both were targeting 100,000 steps; both instead entered a persistent cycle of gradient spikes and watchdog reloads well short of that target, and neither improved further once the cycle began. This repository exists to document why, with checkpoints and audit evidence retained specifically so the failure mode can be studied.
The headline finding, from the SCAF stiffness audit (Phase 7/7b/7c):
The model's own explicit Störmer–Verlet integrator is running past its analytical stability boundary. The anisotropic-Gaussian \(V_\theta\)'s off-diagonal (rank-4 low-rank) curvature pushes the effective per-token stiffness \(\omega \Delta t\) above the Verlet stability bound of 2 at every single audited checkpoint in both runs — while a naive diagonal-only proxy for that same curvature stays under 1 throughout and would have reported no problem at all. This is a structural property of the trained potential, not a training-recipe bug, and it is what motivated replacing Verlet with a CfC + BAOAB propagator (unconditionally stable with respect to this curvature by construction) as this architecture's default integrator going forward.
Separately, both checkpoint sets pass SCAF's Phase-1 causal-leak audit cleanly — the instability documented here is a numerical-integration problem, not a causal-leak regression.
This model is from the Semantic Simulation framework.
Use this repository if you want to:
Do not use this repository if you want a well-trained OpenWebText language model: both checkpoint sets stalled between 7% and 17% of their 100,000-step target, and neither reflects converged behaviour. For a well-trained checkpoint from an earlier, isotropic- \(V_\theta\) version of this architecture, see semsimula-fock-parflm-depthcond-vtheta-openwebtext (27.23 PPL, 250K steps). The anisotropic- \(V_\theta\) + CfC-BAOAB successor run that this repository's finding motivated is expected to supersede both once it completes.
Identical Fock-PARFLM v2.1 scaffold to the d=384 gamma sweep that selected these damping coefficients — same width, depth, \(V_\theta\) family, and Fock register configuration, now run to (attempted) full length instead of a 3,000-step sweep candidate:
Input tokens x_1, ..., x_T
|
Untied token embedding E[x] + learned positional P[t]
|
For each of L=16 damped Störmer–Verlet integration steps (shared force field):
|
+-- K=5 causal-EMA context channels:
| xi^(m)_t = causal_ema(h, alpha_m) [horizons ~2 .. ~200 tokens]
|
+-- Depth-conditioned multi-context V_theta (Anisotropic Gaussian):
| xi_g^(m) = xi^(m) + e_g^(m) [per-layer depth code]
| diff_k^m = h - mu_k^m(xi_g^(m))
| V_m = -sum_k w_k^m exp(-0.5[a_k^m . diff_k^{m2} + ||B_k^{mT} diff_k^m||^2])
| V_theta = sum_m V_m(xi_g^(m), h) [5 contexts, 40 wells total]
| f_theta = -analytical_grad_h V_theta [closed-form, bounded]
|
+-- Sparse pairwise V_phi (structural-competitive, 4 heads):
| top-k=16 past tokens per query (Gumbel routing)
| f_phi = -grad_h V_phi(h_t, h_s) [autograd, sparse]
|
+-- Fock register pool (v2, 32 registers):
| M=32 virtual registers, Q/K/V creation gates, d_k=64
| LIFO stack discipline, per-register tau/keys, ortho init
| register repulsion (Gram penalty, lambda=0.05)
| reverse channel (per-layer, stabilised, pre-LN, soft-norm, warmup 4000)
| prefix-causal (leak-free by construction)
| f_fock = creation + destruction + exchange forces
|
+-- Total force: f = f_theta + f_phi + f_fock
|
+-- Stormer-Verlet step: h += (h-h_prev)/(1+dt*gamma) + dt^2*f/(m*(1+dt*gamma))
|
+-- LayerNorm(h)
|
Logits = h @ W_out^T + b_out [UNTIED W_out]
Auxiliary training-only loss term (not part of the forward pass above):
L_fock_coupling = -lambda_fock * sum_k log(alpha_k + eps) [log-barrier on xi coupling]
| Parameter | Value |
|---|---|
| Hidden dim (d) | 384 |
| Layers (L) | 16 |
| Max sequence length | 1024 (trained at block 512) |
| Vocab (GPT-2 BPE) | 50257 |
| V_theta kind | Depth-conditioned multi-context anisotropic Gaussian (bounded mixture) |
| V_theta contexts (n_ctx) | 5 |
| Wells per context (K) | 8 |
| Total attractors | 40 |
| Anisotropic rank (r) | 4 |
| Precision init / cap | a_k init at -log(384), capped at 2/384 |
| Xi channels (K_xi) | 5 (alpha inits 0.50, 0.75, 0.95, 0.99, 0.995) |
| Fock-coupling regulariser | log-barrier on alpha_k, lambda=0.005, eps=1e-6 |
| V_phi kind | structural_competitive, 4 heads |
| Top-k (sparse routing) | 16 |
| Fock version | v2, 32 registers, d_k=64, LIFO stack |
| Reverse channel | Yes — per-layer, stabilised, pre-LN, soft-norm, 4,000-step warmup |
| Integrator | Störmer–Verlet, position form (dt=1.0) — see below for why this is the finding |
| Embeddings | Untied (separate W_out) |
| Mass model | logfreq (frozen OpenWebText surprisal lookup) |
| Prefix-causal registers | Yes — both checkpoint sets trained natively leak-free from step 0 |
| Total parameters | 77,082,733 |
Every audit result quoted in this card comes from SCAF (SemSimula Causal Auditing Framework), a standalone, model-agnostic library that resolves onto a checkpoint by structural (duck) typing — no base class or Protocol required — and offers two independent audit families:
main): detects, sizes, and attributes causal leaks — any path by which a token at position s influences a prediction at an earlier position t < s, which would otherwise report a perplexity the model did not earn. Two probes ask deliberately different questions: future_perturbation (bit-exact — is there a leak at all?) and target_relocation (how many nats of perplexity were unearned?), backed by three controls (determinism, placebo, positive) that a verdict of CLEAN cannot be issued without. A mediation diagnostic then attributes a confirmed leak to the responsible component by knockout, and scaf.LeakMonitor runs the same probes continuously during training, since leak channels are often gated by a scale initialised at zero and open gradually as training proceeds. A build_leak_frame / estimate_leak bridge hands the same intervention to DoWhy/EconML for exact paired-inference ATEs, refutation tests, and CATE-by-distance heterogeneity profiles via a causal forest. This is what certified both this repository's checkpoint sets CLEAN in SCAF Causal-Leak Audit: Clean.scaf.StiffnessProbe, currently on the stiffness_audit branch, scheduled to merge into main alongside SCAF's other active branches): audits a trained model's numerical integration stability rather than its causal structure, via the diagonal-proxy / Weyl-inequality-upper-bound / native-trajectory-cross-check methodology used throughout this card's stiffness audit section. It is trajectory-faithful by construction (it drives the model through InterventableModel.batch_logits_with_trajectory rather than re-deriving a trajectory by hand), reports block-bootstrap 95% CIs on every fraction-unstable statistic, and skips loudly rather than silently if a model does not expose the harmonic_terms()/mass() hooks it needs.Full design rationale and worked examples for both audit families: docs/Stiffness_Audit_Requirements_and_Design.md and docs/Example_Stiffness_Audit_OWT_g0.1_Anisotropic_Gaussian.md (both on the stiffness_audit branch as of this card's creation; once merged, the same paths resolve on main).
Both runs used identical architecture, data, and optimisation recipe, differing only in \(\gamma_{\text{train}}\), the damping coefficient selected by the gamma sweep ( \(\gamma = 0.10\) was the sweep winner; \(\gamma = 0.30\) was audited as a second candidate on the same architecture to see whether heavier damping alone would suppress the instability). Neither did:
| \(\gamma = 0.10\) | \(\gamma = 0.30\) | |
|---|---|---|
| Target steps | 100,000 | 100,000 |
| Max step actually logged | 17,100 | 11,254 |
| Watchdog reloads | 15 | 5 |
| First reload | step 8,925 | step 7,124 |
Best checkpoint (canonical _best.pt) | step 10,000 | step 10,000 |
| Best val PPL | 184.11 | 211.63 |
| PPL at last logged step | worse than best (192.53 at step 15,000) | worse than best (221.35 at step 7,500) |
Both runs show the same qualitative signature: PPL improves normally up to the first watchdog reload cluster, then stops improving — every checkpoint saved after the reloads begin is worse than the pre-cluster best, and the run never recovers a new best afterward within the logged window. Heavier damping ( \(\gamma = 0.30\) ) delayed the first reload (step 7,124 vs. 8,925 — comparable, if anything slightly earlier) and reduced the reload count (5 vs. 15) but did not eliminate the pattern, which is consistent with a structural integrator problem rather than an under-damping problem (see below).
Both checkpoint sets were run through SCAF's Phase-1 causal-leak audit (FockAdapter, future_perturbation and target_relocation probes, plus determinism/placebo/positive controls). Verdict: CLEAN for both, at every audited checkpoint:
| Probe / control | \(\gamma=0.10\) | \(\gamma=0.30\) |
|---|---|---|
control_determinism | PASS (0.0 logit) | PASS (0.0 logit) |
control_placebo | PASS (0.0 logit) | PASS (0.0 logit) |
control_positive | PASS (9.31 logit — probe reaches the model) | PASS (8.80 logit) |
future_perturbation | PASS (0.0 logit, bit-exact) | PASS (0.0 logit, bit-exact) |
target_relocation | PASS (0.0 nats, PPL inflation 1.0x) | PASS (0.0 nats, PPL inflation 1.0x) |
Both checkpoint sets were trained natively with prefix_causal_registers=True from step 0, so this is the expected result, not a new fix — it confirms the instability documented below is purely a numerical-integration problem, with no interaction with the register-leak fix from earlier in this model family. Full audit output: results/gamma_0.10/scaf_analysis_results/phase1_scaf_audit.json and results/gamma_0.30/scaf_analysis_results/phase1_scaf_audit.json.
The anisotropic-Gaussian \(V_\theta\)'s force is a sum of Gaussian bumps with non-diagonal curvature:
$$ f(h) = -\sum_k g_k(h) P_k (h - \mu_k), \qquad g_k(h) = w_k \exp\left(-\tfrac{1}{2} \mathrm{diff}_k^\top P_k \mathrm{diff}_k\right) > 0, \qquad P_k = \mathrm{diag}(a_k) + B_k B_k^\top, $$
where \(B_k \in \mathbb{R}^{384 \times 4}\) is the learned low-rank factor giving each well its off-axis curvature. Near a well, this is locally a spring with stiffness matrix \(\sum_k g_k P_k\), and the explicit Störmer–Verlet integrator that this model was trained with is a second-order symplectic method whose stability for a harmonic oscillator with frequency \(\omega\) requires
$$ \omega \Delta t < 2 . $$
Above this bound the discrete update's amplification factor leaves the unit circle and every subsequent step amplifies the deviation instead of oscillating around it — trading a bounded, physically faithful trajectory for one that diverges geometrically, which is exactly the gradient-spike-then-reload signature seen in both training logs.
SCAF's stiffness audit (Phase 7) samples the trained model's own hidden-state trajectory and computes \(\omega \Delta t\) at each sampled (token, layer) position, in two ways:
k_diag): keeps only the diagonal of \(\sum_k g_k P_k\), the quantity the CfC+BAOAB propagator (see below) can integrate exactly in closed form.| step | val PPL | diag max | diag frac(>2) | Weyl max | Weyl frac(>2) |
|---|---|---|---|---|---|
| gamma=0.10, 9,000 | 194.68 | 0.6319 | 0.000e+00 | 4.2452 | 5.28e-02 |
| gamma=0.10, 9,500 | 191.84 | 0.3423 | 0.000e+00 | 2.7128 | 5.62e-02 |
| gamma=0.10, 10,000 | 184.11 | 0.4711 | 0.000e+00 | 2.9264 | 5.93e-02 |
| gamma=0.10, 15,000 | 192.53 | 0.4182 | 0.000e+00 | 2.6572 | 6.21e-02 |
| gamma=0.30, 4,500 | 222.60 | 0.4131 | 0.000e+00 | 2.5318 | 5.61e-02 |
| gamma=0.30, 5,000 | 218.08 | 0.5607 | 0.000e+00 | 3.0960 | 2.68e-02 |
| gamma=0.30, 6,000 | 215.09 | 0.4105 | 0.000e+00 | 2.8634 | 2.84e-03 |
| gamma=0.30, 7,500 | 221.35 | 0.5914 | 0.000e+00 | 3.6095 | 2.94e-03 |
| gamma=0.30, 10,000 | 211.63 | 0.3835 | 0.000e+00 | 2.8538 | 3.76e-03 |
At every one of the nine audited checkpoints across both runs, Weyl max clears the stability bound of 2 (range 2.53-4.25), while diag max never even approaches 1. Statistically, Weyl frac(>2) — the fraction of all sampled (token, layer) positions where the Weyl bound itself exceeds 2 — is consistently 2.7%-6.2% of positions, not a rare tail event: a non-trivial share of the model's own forward pass is operating past this integrator's stability boundary at every checkpoint examined.
A synthetic verification (500 trials of random 8-well anisotropic mixtures at d=16, rank=4, checked against the exact eigendecomposition of the true effective matrix) confirms this is a real property of the inequality, not a fluke of these two checkpoints: k_diag's per-dimension maximum underestimated the true top eigenvalue in all 500/500 trials (mean gap 3.77, on a typical eigenvalue scale of 5-18), while the Weyl bound never once fell below the true value. The rank-4 low-rank correction \(B_k B_k^\top\) can concentrate curvature along a direction that is not axis-aligned, and a purely diagonal statistic is structurally blind to exactly that direction — which is precisely the direction the model's own learned wells are using. Full derivation and the synthetic verification plot: Weyl-inequality upper bound section of the SCAF stiffness-audit design doc.
Phase 7 samples the audited trajectory under a forced baoab_cfc integrator regardless of a checkpoint's own native integrator, which could in principle change which curvature the audit samples. Phase 7c closes this caveat by re-running each checkpoint under its own native Verlet trajectory instead, for both runs:
| step | Weyl max, forced | Weyl max, native | Weyl frac(>2), forced | Weyl frac(>2), native |
|---|---|---|---|---|
| gamma=0.10, 9,000 | 4.2452 | 3.0195 | 5.282e-02 | 5.276e-02 |
| gamma=0.10, 9,500 | 2.7128 | 4.0838 | 5.619e-02 | 5.621e-02 |
| gamma=0.10, 10,000 | 2.9264 | 2.7457 | 5.932e-02 | 5.932e-02 |
| gamma=0.10, 15,000 | 2.6572 | 3.1421 | 6.206e-02 | 6.204e-02 |
| gamma=0.30, 4,500 | 2.5318 | 2.4708 | 5.610e-02 | 5.608e-02 |
| gamma=0.30, 5,000 | 3.0960 | 2.6971 | 2.683e-02 | 2.673e-02 |
| gamma=0.30, 6,000 | 2.8634 | 2.7393 | 2.838e-03 | 2.777e-03 |
| gamma=0.30, 7,500 | 3.6095 | 2.8911 | 2.945e-03 | 2.914e-03 |
| gamma=0.30, 10,000 | 2.8538 | 2.8538 | 3.761e-03 | 3.754e-03 |
Weyl frac(>2) agrees between forced and native trajectories to within 0.02 percentage points at every single checkpoint in both runs, and Weyl max stays clear of the stability bound in both modes at every checkpoint too. The forced-vs-native discrepancy is negligible next to the trend it is being asked to corroborate — the trajectory-substitution caveat is closed for both damping coefficients, by their own checkpoints, without needing extra training.
This evidence — a structural curvature violation of the Verlet stability bound present at every audited checkpoint, in both damping regimes, confirmed independent of the audit's own trajectory choice — is what motivated replacing Störmer–Verlet with a Closed-form Continuous-time (CfC) + BAOAB propagator for this architecture going forward. CfC integrates the harmonic component of the force in closed form (via torch.sinc-based exact-flow update rather than an explicit finite-difference step), which is unconditionally stable with respect to this curvature by construction rather than conditionally stable below \(\omega \Delta t = 2\).
This is not a free lunch: the CfC+BAOAB propagator costs roughly 2.4x Störmer–Verlet's per-step wall-clock time on the same GPU. An early CfC+BAOAB run also hit its own regression — a grad=nan traced to torch.sqrt's infinite derivative at exactly k_diag/m == 0 inside cfc_substep — which was fixed with an _OMEGA_SQ_FLOOR = 1e-12 clamp and confirmed to produce zero NaNs across a full resumed training run. The CfC+BAOAB run is in progress as of this card's creation and will be published as its own model card once complete; see the deep-dive writeup for the full implementation: PyTorch_Implementation_of_CfC_BAOAB_in_Fock-PARFLM.md.
import math, torch, sys
sys.path.insert(0, "multixi")
sys.path.insert(0, "parf")
sys.path.insert(0, "energetic_minima")
sys.path.insert(0, "sarf_mass_variant")
from parf.model_fock_parf_multixi import FockMultiXiPARFLM, FockMultiXiPARFConfig
from parf.model_aniso_gaussian_vtheta import AnisotropicDepthConditionedGaussianVTheta, install_aniso_depth_routing
from huggingface_hub import hf_hub_download
REPO = "dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-verlet-instability"
GAMMA = "0.10" # also available: "0.30"
STEP = 10000 # the canonical best checkpoint for either gamma
logfreq_path = hf_hub_download(repo_id=REPO, filename=f"results/gamma_{GAMMA}/logfreq_surprisal_openwebtext.npy")
config = FockMultiXiPARFConfig(
vocab_size=50257, d=384, max_len=1024, L=16,
mass_mode="logfreq", logfreq_path=logfreq_path,
init_gamma=1.0, fixed_gamma=float(GAMMA),
xi_channels=5, xi_alpha_inits=[0.50, 0.75, 0.95, 0.99, 0.995],
xi_learnable=True, xi_alpha_init_mode="explicit",
fock_version="v2", n_registers=32,
reverse_channel=True, reverse_channel_stable=True, reverse_channel_pre_ln=True,
reverse_channel_soft_norm=True, reverse_channel_warmup_steps=4000, reverse_channel_per_layer=True,
register_repulsion=True, register_repulsion_coeff=0.05,
prefix_causal_registers=True,
v_phi_kind="structural_competitive", v_phi_n_heads=4, v_phi_d_type=32, v_phi_d_angle=16, top_k=16,
use_output_bias=True, tie_embeddings=False,
per_register_tau=True, per_register_keys=True, ortho_register_init=True,
)
model = FockMultiXiPARFLM(config)
model.V_theta = AnisotropicDepthConditionedGaussianVTheta(
d=384, K=8, n_ctx=5, n_layers=16, rank=4,
w_scale=1.0,
init_log_precision=-math.log(384),
precision_max=2.0 / 384,
code_init_std=0.02,
)
install_aniso_depth_routing(model)
ckpt_path = hf_hub_download(repo_id=REPO, filename=f"checkpoints/gamma_{GAMMA}/ckpt_step{STEP}_best.pt")
state = torch.load(ckpt_path, map_location="cpu", weights_only=False)
model.load_state_dict(state["model_state_dict"])
model.eval()
print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}") # 77,082,733
print(f"gamma={state['gamma']} step={state['step']:,} val_ppl={state['val_ppl']:.2f}")
| Path | Description |
|---|---|
checkpoints/gamma_0.10/ckpt_step{9000,9500,10000}.pt (_best), ckpt_step15000.pt | The four gamma=0.10 checkpoints audited by SCAF, plus the last periodic checkpoint of the run |
checkpoints/gamma_0.30/ckpt_step{4500,5000,6000}.pt (_best), ckpt_step{7500,10000}.pt | The five gamma=0.30 checkpoints audited by SCAF (step 10,000 is also _best) |
checkpoints/gamma_*/training_log.jsonl | Full per-step training log, including every watchdog_reload and grad_spike event |
results/gamma_*/logfreq_surprisal_openwebtext.npy | Frozen per-token log-frequency (surprisal) lookup used by the logfreq mass model, per run |
results/gamma_*/scaf_analysis_results/phase1_scaf_audit.json | SCAF Phase-1 causal-leak audit (verdict, controls, probes) |
results/gamma_*/scaf_analysis_results/phase7_stiffness_audit.json | Diagonal-proxy omega*dt statistics per checkpoint |
results/gamma_*/scaf_analysis_results/phase7b_eig_bound_audit.json | Weyl-bound omega*dt statistics per checkpoint, with block-bootstrap 95% CIs |
results/gamma_*/scaf_analysis_results/phase7c_native_trajectory_audit.json | Native-Verlet-trajectory cross-check of both the diagonal proxy and the Weyl bound |
results/gamma_*/scaf_analysis_results/*.png | All audit figures (stiffness-vs-step, eig-bound-vs-step, phase7c native-vs-forced) |
results/gamma_*/scaf_analysis_results/phase2*.json, phase2_5*.csv, phase3*.json, phase4*.json, phase5*.png, phase6*.{json,png} | Supporting SCAF causal-estimation phases (ATE, register diagnostics, CATE-by-distance), included for completeness |
model_aniso_gaussian_vtheta.py | Anisotropic Gaussian V_theta classes + install_aniso_depth_routing (identical file to the gamma-sweep repo's) |
config.json | Full run configuration, per-run summary statistics, and links to the successor CfC+BAOAB work |
OpenWebText, tokenized with GPT-2 BPE (vocab 50257), evaluated on a held-out 2M-token validation slice with no train/val overlap. Both runs targeted 100,000 steps and were early-terminated (in the sense of "checkpoints stopped being retained as _best") by the instability documented above, at step 17,100 (gamma=0.10) and step 11,254 (gamma=0.30).
| Hyperparameter | Value |
|---|---|
| Optimizer | AdamW (betas 0.9/0.95, weight decay 0.01) |
| LR schedule | WSD |
| Peak learning rate | 3e-4 |
| Block size | 512 |
| Target steps | 100,000 |
| Gradient clipping | per-group (global 1.0; V_phi and Fock gates tighter) |
| Watchdog | reloads last-best checkpoint on gradient-norm-EMA or grad-spike triggers |
| lambda_V (V_theta regularisation) | 0.01 |
| lambda_fock (coupling regulariser) | 0.005 |
| Hardware | 1x NVIDIA H100/A100 (Google Colab) |
See SCAF Causal-Leak Audit: Clean above — both runs pass with verdict CLEAN.
notebooks/conservative_arch/scaleup/colab_fock_aniso_gaussian_fockreg_openwebtext.ipynb (companion repo). SCAF audit performed with notebooks/conservative_arch/scaleup/debug/scaf_checkpoint_analysis.ipynb (same repo).
| \(\gamma\) | Best step | PPL |
|---|---|---|
| 0.10 | 10,000 | 184.11 |
| 0.30 | 10,000 | 211.63 |
Neither of these is comparable to the fully trained OpenWebText checkpoints elsewhere in this family (e.g. 27.23 PPL after 250K steps for the isotropic flagship) — both runs stalled at 10-17% of their 100,000-step target due to the instability documented above.
This model is part of the Semantic Simulation SPLM family:
| Model | Design | Corpus | PPL | HuggingFace |
|---|---|---|---|---|
| Multi-Xi SPLM (MLP) | Pure scalar potential | TinyStories | 11.51 | semsimula-splm-multixi |
| Multi-Xi SPLM (SQ3) | Structured scalar potential | TinyStories | 13.33 | semsimula-splm-multixi-structured-vtheta |
| Multi-Xi PARFLM (MLP) | Scalar + pairwise forces | TinyStories | 12.06 | semsimula-parflm-multixi |
| Multi-Xi PARFLM (SQ3) | Structured scalar + pairwise | TinyStories | 12.27 | semsimula-parflm-multixi-structured-vtheta |
| Fock-PARFLM v2.1 (MLP) | PARFLM + Fock registers | TinyStories | 9.70 | semsimula-fock-parflm |
| Fock-PARFLM v2.1 (SQ3) | Structured + pairwise + Fock | TinyStories | 10.90 | semsimula-fock-parflm-structured-vtheta |
| Fock-PARFLM v2.1 (depth-cond. isotropic Gaussian) | Bounded multi-context + pairwise + Fock | TinyStories | 16.33 | semsimula-fock-parflm-depthcond-vtheta |
| Fock-PARFLM v2.1 (depth-cond. anisotropic Gaussian + fock-reg) | Bounded, ellipsoidal multi-context + pairwise + Fock | TinyStories | 9.04 | semsimula-fock-parflm-anisogaussian-vtheta |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, first-order/Fock-G1) | Same as above, gradient-flow integrator | TinyStories | 8.95 | semsimula-fock-parflm-anisogaussian-vtheta-fock-g1 |
| Fock Attention (MLP V_theta) | Fock + attention | TinyStories | 9.42 | semsimula-fock-attention |
| Hybrid SPLM+Attn | Attention + SPLM refinement | TinyStories | 8.50 | semsimula-hybrid-splm |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=384) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 278.27 (best of 8, 3K-step sweep) | semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=768) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 326.97 (best of 8, 3K-step sweep) | semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=1024) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 244.23 (best of 8, 3K-step sweep) | semsimula-fock-parflm-anisogaussian-vtheta-owt-d1024-gammasweep |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, Verlet instability, d=384) | Same architecture, two full-run attempts — SCAF stiffness audit identifies structural Verlet instability | OpenWebText | 184.11 / 211.63 (both runs stalled, not final) | this repository |
Collection: Semantic Simulation SPLM Model Family
Weyl frac(>2) values (2.7%-6.2%) are a ceiling on how much of each run crosses the stability line via the off-diagonal direction, not an exact measurement. The exact-eigenvalue extension (torch.linalg.eigvalsh on the small rank-4-derived matrix) is a documented follow-up, not yet run against these checkpoints.prefix_causal_registers=True; see SCAF Causal-Leak Audit: Clean.@misc{Gueorguiev2026SemSim,
author = {Gueorguiev, Dimitar P.},
title = {Semantic Simulation: A Prescriptive Lagrangian Framework
for Efficient Semantic Inference --- A Conservative-by-
Construction Language Model and the Shared-Potential
Separator, with a Correspondence to Joint Embedding
Predictive Architectures},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.19712427},
url = {https://doi.org/10.5281/zenodo.19712427},
note = {Companion code repository:
\url{https://github.com/dimitarpg13/semsimula-paper}}
}
Correction (2026-09-27) — "velocity-Verlet" was the wrong name. This lineage's layer step carries no velocity state: the velocity is the position difference
h - h_prev, and the update ish_new = h + (h - h_prev)/(1 + dt*gamma) + dt^2*f/(m*(1 + dt*gamma)), which undamped ish_{n+1} = 2*h_n - h_{n-1} + dt^2*f/m. That is Störmer–Verlet in position form. Velocity-Verlet is the variant that does carry an explicit velocity, through half-kick/drift/half-kick, and was never used here. The \(\omega \cdot dt < 2\) stability bound is the Störmer/leapfrog bound and is unaffected by the correction, as is every measurement on this card.