Downloads · 30 days
0
AcidAI/Acid-GSN-Architecture
Acid-GSN-Architecture is a text generation model from AcidAI. Use it when you need the model to write or continue text. It is set up for custom. The card lists the license as other.
Downloads · 30 days
0
Access
Public
Updated Jul 24, 2026
Repo size
15.2 KB
Likes
0
Public
Click a slice to open those files.
.zip15.2 KB · 53%
From the Hugging Face model README
Acid Research (ACRS)
GSN is a language model architecture designed around one principle: do not spend compute you do not need.
Unlike transformers — which run the full model on every input regardless of complexity — GSN gates compute dynamically. Simple inputs take a shallow path. Complex inputs go deeper. The network decides, not the configuration.
GSN is built from three core ideas stacked together:
1. Gated Depth A lightweight gate network evaluates each input and decides how many layers to activate. Layers that are not needed do not run. Their compute cost is exactly zero.
2. Sparse Activation Within each active layer, only the top-k neurons fire. The rest are masked to zero. A 4-layer GSN running at 25% sparsity uses a fraction of the compute a dense model would.
3. Recurrent Encoder Input tokens are processed sequentially through a GRU encoder before the gate sees anything. This replaces mean pooling — token order matters, context accumulates, and the gate receives a hidden state that actually encodes sequence structure.
4. Compute Penalty in Training The loss function penalizes wasted compute. The model is trained to be cheap, not just accurate. Over time the gate learns: if I can answer correctly with one layer, using two is a mistake.
No attention. No transformers. O(T) encoding. O(1) per sparse layer.
| Property | Transformer | RSM | GSN |
|---|---|---|---|
| Attention cost | O(n²) | none | none |
| Compute per input | fixed | fixed | dynamic |
| Sparse activation | no | no | yes |
| CPU viable | marginal | yes | yes |
| Trains on 2 cores | no | hours | minutes |
GSN was developed and trained entirely on a consumer CPU with 2 physical cores. No GPU. No cloud compute. That is not a limitation — it is the point.
pip install numpy
No PyTorch. No CUDA. No framework dependency. Pure NumPy.
python train.py your_corpus.txt 20000
Point it at any plain text file. The tokenizer trains from scratch on your corpus. Checkpoints save every 500 steps. Resume is automatic.
python infer.py "your prompt here" --max_new 100
The inference report shows complexity score, gate depth decision, and compute saved per generation.
All hyperparameters live in config.py. Key settings:
GSNConfig(
vocab_size = 1024, # BPE vocabulary size
dim = 256, # embedding and hidden dimension
enc_dim = 256, # GRU encoder hidden dimension
n_layers = 4, # total sparse layers available
compute_penalty = 0.001, # λ — weight of compute cost in loss
lr = 3e-4, # learning rate
seq_len = 128, # context length
)
Corpus: Shakespeare complete works (~1.1M characters)
Vocabulary: 1024 BPE tokens
Steps: 20,000
Hardware: 2 physical CPU cores (Debian Linux)
Training time: ~25 minutes
Final loss: ~3.5
Average gate depth: 1.44 / 4 layers
Average compute saved: ~88%
The gate learned that Shakespeare — structured, repetitive, consistent vocabulary — is mostly shallow complexity. On a more diverse corpus the gate is expected to show greater depth variation.
config.py — all hyperparameters
tokenizer.py — BPE tokenizer, trains from scratch
model.py — GSN model, forward pass, analytical backprop
train.py — training loop, Adam optimizer, checkpointing
infer.py — inference CLI with compute report
This is an incomplete release by design. The architecture is sound. The community is invited to take it further.
Licensed under the Acid Research Protected Interests License (APRIL) v1.0.
See LICENSE for full terms.
If you use GSN in research or build on this architecture, please cite:
@misc{gsn2025,
title = {GSN: Gated Sparse Network},
author = {Acid Research (ACRS)},
year = {2025},
url = {https://huggingface.co/AcidAI/Acid-GSN-Architecture}
}
Acid Research (ACRS) is an independent AI research organization building CPU-native, economically viable alternatives to transformer-based architectures.
Current architecture portfolio:
"Make it linear, or else your cost ain't going to be."