Sparse-AST-BWM-3M-32

Sparse-AST-BWM-3M-32

Ultra-lightweight 4.18M Sparse-AST model designed for low-latency syntax drafting and basic Blender Python token prediction.

Model Details

• Architecture: Sparse-AST (Recurrent Leaky Gated Transformer)
• Parameters: 4,177,488
• Serialization: SafeTensors (model.safetensors)
• Tied Embeddings: Yes (e.weight tied to h_out.weight, preserved via SafeTensors metadata)
• Vocabulary: 512 (UTF-8 byte-level tokenization)
• Domain: Procedural 3D Mathematics, Blender Python (bpy, mathutils, bmesh, numpy, gpu)

Context Window Specification

• Native Training Context: 64 tokens
• Inference-Time Context Interpolation: Supported up to 4,096 tokens via 1D linear positional interpolation in from_pretrained(..., target_context=4096).

[!NOTE] This model was trained natively with a 64-token context window. For longer sequences, positional interpolation is applied at inference time.

Experimentally Measured Benchmarks

Evaluated on the standardized Blender 3D Math & Python Curriculum suite:

Verification & Numerical Integrity

This SafeTensors distribution underwent full CPU verification against the original PyTorch checkpoint:

• Checked Tensors: 72
• Tied Weights Handled: 1
• Max Absolute Error: 0.0 (Exact 0.0 bitwise equality)
• Verification Status: PASS

Quick Start Inference

from model import SparseAST
model = SparseAST.from_pretrained('.')
from generate import generate
prompt = 'import mathutils\nfrom mathutils import Vector\nv = Vector(('
output = generate(model, prompt, max_new_tokens=30)
print(output)