AGILLM-4.4

AGILLM-4.4 — Detachable 49.1M AR Student

This repository releases a point-in-time, model-only export of the 49,097,218-parameter detachable causal-language-model student produced alongside the continuing 1,221,580,802-parameter AGILLM parent.

The initial weight release does not contain the 1.22B parent weights or parent optimizer state. The released weight artifact was frozen from the student state associated with parent checkpoint step 2,560,770. The current single-file trainer source is now published below.

Adaptive General Intelligence Large Language Model v4.4 is the release boundary for AGILLM's checkpoint-compatible, Matryoshka-inspired teacher-student system:

• a mature 1,221,580,802-parameter parent that continues its existing AR/SAT/NAT DiffusionBlock training; and
• a separately optimized, independently exportable 49,097,218-parameter causal-AR student trained from detached parent representations.

AGILLM 4.4 does not replace or freeze the parent. The parent keeps training normally. After an accepted parent update, a stop-gradient copy of an already-computed mature AR hidden representation may train one attention or FFN sublayer in the detachable student.

accepted mature 1.22B DBlock commit
              |
              +-- detached final AR hidden state
                           |
                           +-- one 49.1M student sublayer update

The student has no authority to select parent batches, route parent layers, change parent gradients, retry a mature batch, or inject activations into the mature forward pass.

Current single-file trainer — 12 September 2026

Download agillm44_singlefile.py

This is the full current trainer/inference/supervisor source. All AGILLM custom Python code is in this one file, including eight embedded support modules and the SAT seed tensors. It preserves the running trainer's AR/SAT/NAT, DBlock and dynamic SAT-var code.

Python and third-party libraries remain required; use requirements-trainer.txt for the verified Python 3.10 environment. Checkpoints, tokenizer/data assets and optional run configuration are inputs rather than additional Python modules.

python agillm44_singlefile.py --help
python agillm44_singlefile.py train --help

Source SHA-256: a0660f15ec322f815e8133aa116e60c693d4fbd9f48ee4b232432ae8c71ffc50.

The source and embedded modules were checked for literal credentials. No hardcoded credentials were identified; private credential files and environment values are not included. Proxy URLs are redacted in supervisor logs. See runtime/README.md.

Matryoshka inspiration

This work is inspired by Matryoshka Language Model Suites, but it is not a faithful reproduction of that paper's shared-parameter, end-to-end nested suite.

AGILLM-4.4 retrofits a separate top-level student around an already mature parent. The student was initialized from deterministic read-only slices of the parent and then trained with detached teacher representations. Results reported for the referenced paper—including benchmark parity, compute savings, or speculative-decoding gains—are not results for this implementation.

Why this is 4.4

AGILLM 4.3 established the mature 28-layer DiffusionBlock/MoE training line. AGILLM 4.4 adds a new checkpointed component with its own architecture, optimizer, scaler, RNG, counters, schedule, export format, and failure boundary. Internal agillm43.* checkpoint identifiers are intentionally retained where required for backward compatibility; the product/release boundary is 4.4.

Architecture

The standalone file is below 50 million parameters. The bridge is training-only and is excluded from the independently usable export.

Initial public artifact

The first AGILLM 4.4 repository artifact is exported from the atomic joint checkpoint below:

Artifact:

artifacts/detachable-50m/detachable_50m_step02560770.pt

• Size: 182,879,879 bytes
• SHA-256: 46103c67118eadb6a265c3c8ba871b6d4732344c567ee5247af224f22e0f8a4a
• Exact unique parameter count: 49,097,218
• Embedded tokenizer: restored successfully
• ALiBi contract: corrected, scale 0.0
• Native AGILLM AR loader test: passed on CPU and generated one token

The one-token loader result is a compatibility smoke test, not a quality or throughput benchmark. The parent observed-token counter belongs to the parent; it is not a claim that the student directly consumed 248B tokens.

Inference

This is a custom AGILLM checkpoint, not a transformers.AutoModelForCausalLM directory. Use the included model-only loader:

python agillm44_detachable_50m_loader.py \
  --checkpoint artifacts/detachable-50m/detachable_50m_step02560770.pt \
  --expected-sha256 46103c67118eadb6a265c3c8ba871b6d4732344c567ee5247af224f22e0f8a4a \
  --expected-tokenizer-sha256 0e546c51529290ed4b18dc8b3e481a9ea39a2df2175b65ecb281af6bac79af73 \
  --device cpu \
  --prompt "The capital of France is" \
  --max-new-tokens 64 \
  --greedy

The loader contains only the detachable model definition, bounded zstd decoding, exact contract validation, embedded-tokenizer restoration, and AR generation. Both the outer wrapper and decompressed payload use torch.load(..., weights_only=True) with no unsafe fallback. It has no training, network, credential, supervisor, checkpoint-retention, or production-provenance code.

The current production trainer source is available as agillm44_singlefile.py. It includes training, inference, supervision, checkpoint handling and all custom modules. See runtime/README.md for dependencies and the source audit.

The verified public-loader test used Python 3.10, PyTorch, tokenizers==0.21.0, and zstandard support. The historical production loader also passed with transformers==4.48.0 and tokenizers==0.21.0. GPU training additionally uses bitsandbytes/PagedAdamW8bit.

What is and is not established

Established:

• the mature parent continues training while the student trains;
• teacher tensors are stop-gradient and transient;
• parent parameters are absent from the student optimizer;
• a student failure cannot retry or undo an accepted mature update;
• parent and student state can be saved together and the student can be exported independently;
• the exported artifact passes the normal AGILLM AR load path.

Not established:

• no claim is made yet that parent-guided training improves held-out quality per GPU-second versus a matched student-only baseline;
• train-crop CE is not a held-out evaluation and should not be treated as a standardized benchmark;
• no broad safety, factuality, instruction-following, or production-serving evaluation has been completed;
• with the calibrated SAT-VAR gate attached, the standalone student performs dynamic variable-stride (1 vs 2 tokens) generation balancing speed and intelligence;
• hackathon or competition eligibility is a separate rules question.

A decrease in online hidden-distillation training loss is telemetry, not a held-out evaluation or an acceleration result.

Checkpoints and repository scope

This repository includes the standalone student, the model-only loader, tokenizer bundle, release metadata, documentation and the current single-file trainer source. The original joint parent/student training checkpoint is not duplicated into this repository; see CHECKPOINTS.md. Publishing the current trainer source does not replace the older weight artifacts.

License

The repository uses the same license: other classification as the existing AGILLM 4.3 model repository. No additional license grant is implied by publication; review the repository terms before redistribution or commercial use.

SAT-VAR Variable Stride (1-vs-2 Token) Dynamic Inference

AGILLM-4.4 features SAT-VAR (Self-Attentive / Variable Stride Autoregressive generation), dynamically emitting 1 or 2 tokens per forward pass based on a balanced speed-versus-intelligence training objective:

• Speed vs Intelligence Balance: Emitting 2 tokens per forward step doubles decoding throughput for predictable tokens, while falling back to 1 token preserves maximum representation fidelity when semantic uncertainty or complexity is high.
• Calibrated Gate (det50_export_sat_gate_balanced.pt): A 514-parameter linear gate (Linear(256, 2)) trained and calibrated directly on internal hidden representations from the student checkpoint across held-out tokens (achieving AUROC 0.7619).
• Auto-Discovery & Zero Configuration: The loader automatically discovers companion gate files in the checkpoint or script directory, with a default utility breakeven operating threshold of $\tau = 0.45$.

Quickstart with SAT-VAR

# Zero-config: companion gate det50_export_sat_gate_balanced.pt is automatically discovered
python agillm44_detachable_50m_loader.py \
  --checkpoint artifacts/detachable-50m/detachable_50m_step02560770.pt \
  --prompt "Artificial intelligence enables systems to" \
  --satvar \
  --max-new-tokens 64 \
  --device cpu

Or specify the gate and custom threshold explicitly:

python agillm44_detachable_50m_loader.py \
  --checkpoint artifacts/detachable-50m/detachable_50m_step02560770.pt \
  --sat-gate artifacts/detachable-50m/det50_export_sat_gate_balanced.pt \
  --satvar-threshold 0.45 \
  --prompt "The history of computing began when" \
  --satvar \
  --max-new-tokens 64 \
  --device cuda

Telemetry

At inference time, the loader reports real-time execution statistics:

[agillm44] device=cuda dtype=bfloat16 generated=48 elapsed=4.35s satvar=active strides={'1': 14, '2': 17} mean_stride=1.55 tok_per_fwd=0.980

• strides: distribution of single-token (stride 1) vs speculative two-token (stride 2) emissions.
• mean_stride: effective speedup factor across the generated sequence.
• tok_per_fwd: generated tokens per core model forward pass.