Downloads · 30 days
0
holo-llm-hrr-attention
staccs/holo-llm-hrr-attention
holo-llm-hrr-attention is a text generation model from staccs. Use it when you need the model to write or continue text. The card lists the license as mit.
Archive / conversion-track repo. Drop-in HRR attention experiments on Qwen2.5 (swap softmax for a fixed-size holographic state).
Access
Public
Updated Oct 29, 2026
Repo size
681 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt493 MB · 83%
At a glance
- Task
- Text Generation
- License
- mit
- Access
- Public
- Created
- Aug 8, 2026
- Updated
- Oct 29, 2026
- SHA
- 640ec61e
Try a prompt
Model card
From the Hugging Face model README
Holo LLM — HRR Attention Conversions
Archive / conversion-track repo. Drop-in HRR attention experiments on Qwen2.5 (swap softmax for a fixed-size holographic state).
Live Status (2026-08-13)
The conversion-track table below is historical. Those “converting / queued / QLoRA” rows are not live jobs. Do not treat this card as the product.
Current product / live jobs (other repos / boxes):
| What | Where |
|---|---|
| DeepSeek-V4-Flash in-weight HRR overlay (memory 5/5 vs 0/5, KV spill needle) | staccs/lecore-deepseek-v4-flash-hrr |
| Qwen3.5-9B Unicron assimilate prove | staccs/lecore-qwen35-9b-assimilated |
| GPT-2 HRR vs softmax code + small ckpts | staccs/lecore-hrr-gpt2 |
| Qwen3.5-397B Galvatron | Vast 47593358 — not uploaded |
| OLMo 2 from-scratch HRR (original mix) | Vast 47603357 8×A100 — no loss yet, not this repo |
Historical conversion track (stale)
| Model | Status as originally written |
|---|---|
| Qwen2.5-0.5B-Instruct → HRR | Converted, fine-tuning (old) |
| Qwen2.5-7B-Instruct → HRR | Converting (old) |
| Qwen2.5-14B-Instruct → HRR | Queued (old) |
| Qwen2.5-32B-Instruct → HRR | QLoRA fine-tuning (old) |
| Qwen2.5-72B-Instruct → HRR | Queued (old) |
This repo still holds conversion scripts and some Qwen2.5-1.5B / gated-scan checkpoints. That is the artifact. It is not a hosted 72B.
How it was supposed to work
Each attention module is replaced with HRR attention — a gated holographic recurrence that stores context in a fixed-size superposed state. Conversion copies Q/K/V/O projection weights and adds learned per-frequency forgetting gates.
from transformers import AutoModelForCausalLM, AutoTokenizer
# only if the subfolder actually exists in this repo
model = AutoModelForCausalLM.from_pretrained("staccs/holo-llm-hrr-attention", subfolder="qwen0.5b_hrr")
See also
- lecore-deepseek-v4-flash-hrr — current Flash overlay
- lecore-hrr-gpt2 — from-scratch GPT-2 HRR proofs
Files in this repo
| file | size | what it is |
|---|---|---|
cloud/ckpt_Qwen2.5-1.5B_hrr_s1b.pt | 308.7 MB | support file |
data/enwik8 | 100.0 MB | support file |
checkpoints/ckpt_qwen_hrr_stage1b.pt | 88.3 MB | support file |
checkpoints/ckpt_softmax_T1024.pt | 49.6 MB | support file |
checkpoints/ckpt_gated_T1024_a0.9.pt | 46.1 MB | support file |
checkpoints/ckpt_qwen_hrr_stage1.pt | 82.6 KB | support file |
model.py | 14.2 KB | support file |
RESULTS.md | 10.7 KB | support file |
- Task
- Text Generation
- License
- mit
- Languages
- en
- Created
- Aug 8, 2026
- Updated
- Oct 29, 2026