Downloads · 30 days
18
1% of all-time downloads
ethicalabs/Echo-DSRN-114M-v0.1.1
Echo-DSRN-114M-v0.1.1 is a text generation model from ethicalabs. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
[!WARNING] This repository contains experimental models designed strictly for academic evaluation and research purposes. Critical Constraints: No Production Deployment: Experimental models must not be deployed in comm…
Downloads · 30 days
18
1% of all-time downloads
All-time downloads
2.2K
Public
Parameters
115M
459 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors459 MB · 99%
From the Hugging Face model README
[!WARNING] This repository contains experimental models designed strictly for academic evaluation and research purposes.
Critical Constraints:
- No Production Deployment: Experimental models must not be deployed in commercial, enterprise, or mission-critical environments under any circumstances.
- No Liability: Experimental models are provided "as-is" without warranties of any kind. The developers assume zero liability for downstream consequences, system integration failures, or regulatory non-compliance resulting from unauthorized deployment.
The Echo-DSRN(N) (Dual State Recurrent Neural Network, short name: Echo-DSRN, also know as echo) is a novel architecture specifically designed to be a viable alternative for low-resource tasks that are currently being inefficiently handled by the excessive scale of Large Language Models (LLMs) 🌱
This is a research prototype and demo model.
| Property | Value |
|---|---|
| Model Type | echo_dsrn |
| Layers | 8 |
| Hidden Dim | 512 |
| Attention Heads | 4 |
| MLP Ratio | 8.0 |
| Vocab Size | 32011 |
| Hybrid Attention | True |
| RMSNorm | True |
| Component | Parameters | % of Total |
|---|---|---|
| Total | 114.69M (114,687,488) | 100% |
| Embeddings | 16.39M | 14.29% |
| DSRN Blocks (Aggregate) | 81.91M | 71.42% |
| LM Head | 16.39M | 14.29% |
| Sub-Component | Parameters | Description |
|---|---|---|
| MLP (Feed-Forward) | 4.20M | Upscaled hidden layers |
| DSRN Slow State | 3.15M | Constant-time memory gates |
| GRU Fast State | 1.58M | Recurrent fast path |
| Surprise Gating | 264,192 | Dynamic focus mechanism |
| Normalization | 1,024 | LayerNorm / RMSNorm |
Work in progress.