Downloads · 30 days
0
fdra-half-life-regularization
fractal-agi/fdra-half-life-regularization
fdra-half-life-regularization is a machine learning model from fractal-agi. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Date: 2026-01-22 Repository: fractal-agi/fdra-half-life-regularization
Access
Public
Updated Jan 23, 2026
Repo size
404 KB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json978 KB · 57%
At a glance
- Access
- Public
- Created
- Jan 22, 2026
- Updated
- Jan 23, 2026
- SHA
- 70969005
Model card
From the Hugging Face model README
FDRA Half-Life Regularization V3: Bug-Fixed Implementation
Date: 2026-01-22
Repository: fractal-agi/fdra-half-life-regularization
Summary
This package contains the bug-fixed implementation of the half-life regularizer for FDRA oscillators, addressing critical issues discovered during code review.
The Problem
FDRA models trained at GPT-2 scale experience half-life collapse: all oscillator decay parameters converge to short values (τ ≈ 2-10 steps), losing the ability to maintain long-range context.
The Solution
A half-life regularizer that maintains a log-uniform distribution of τ ∈ [1, L] where L is the sequence length, ensuring some oscillators can attend to the full context.
Bug Fixes (V3)
| Bug | Severity | Issue | Fix |
|---|---|---|---|
np.clip argument order | CRITICAL | np.clip(x, max, min) clips everything to min | Swapped to (min, max) |
| Missing tau bounds | CRITICAL | Moment-matching created pathological τ<1 | Added compute_bounds_loss() |
| Sigmoid overflow | Medium | exp(-k*tau) overflowed | Added np.clip(x, -500, 500) |
| Learning rate | Medium | lr=0.3 overshot valid λ range | Changed to lr=0.0001 |
| Mean-only convergence | Medium | All τ converged to same value | Use log-uniform init directly |
Results
Before Bug Fixes (V1-V2)
Regularized tau: [0.48, 6931.1] ← PATHOLOGICAL
23/32 oscillators with τ < 1 ← WORSE than collapsed!
Basin width: 256 tokens
After Bug Fixes (V3)
Regularized tau: [1.0, 4096.0] ← Proper log-uniform spread
3/32 oscillators with τ > 2048 ← Long-range coverage
Basin width: 1024 tokens ← 4x improvement
Identity Reconstruction Comparison
| Condition | Verdict | Basin Width | Notes |
|---|---|---|---|
| Collapsed (no regularization) | FAIL | 0 | Identity immediately lost |
| Regularized (log-uniform τ) | PARTIAL | 1024 (25% of L) | Identity preserved to K=1024 |
Files
├── half_life_regularizer.py # Core regularizer with bounds constraint
├── fdra_oscillators.py # Oscillator bank implementation
├── identity_reconstruction_experiment_v2.py # Fixed diagnostic experiment
├── identity_v2_*.json # Raw experimental results
├── IDENTITY_V2_REPORT_*.md # Generated reports
├── BUGFIX_REPORT.md # Detailed bug analysis
└── IMPLICATIONS.md # Research implications
Usage
from half_life_regularizer import HalfLifeRegularizer, HalfLifeRegularizerConfig
config = HalfLifeRegularizerConfig(
sequence_length=4096,
tau_min=1.0,
tau_max=4096.0,
lambda1=0.01, # Log-uniform prior weight
lambda2=0.01, # Long-tail weight
lambda3=0.1 # Bounds constraint weight (NEW!)
)
regularizer = HalfLifeRegularizer(config)
# During training:
loss, metrics = regularizer.compute(oscillator_lambdas)
total_loss = task_loss + loss
Citation
If you use this work, please cite:
@misc{fdra-half-life-regularization-2026,
title={Half-Life Regularization for FDRA Oscillators: Preventing Decay Collapse},
author={Fractal AGI},
year={2026},
publisher={Hugging Face},
url={https://huggingface.co/fractal-agi/fdra-half-life-regularization}
}
License
Apache 2.0
- Created
- Jan 22, 2026
- Updated
- Jan 23, 2026