Downloads · 30 days
9
7% of all-time downloads
Ouroboros-1MContext-Gemma-270m
MonumentalSystems/Ouroboros-1MContext-Gemma-270m
Ouroboros-1MContext-Gemma-270m is a text generation model from MonumentalSystems. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
1M-context extension of Google's Gemma 3 270M for episodic memory research.
All-time downloads
124
Public
Parameters
268M
570 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors536 MB · 94%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- gemma
- Model type
- gemma3_text
- Access
- Public
- Created
- Mar 19, 2026
- Updated
- Oct 8, 2026
- SHA
- 2ff59ec8
Try a prompt
Base models
Model card
From the Hugging Face model README
Ouroboros-1MContext-Gemma-270m
1M-context extension of Google's Gemma 3 270M for episodic memory research.
Architecture
| Parameter | Value |
|---|---|
| Model type | Gemma3ForCausalLM |
| Hidden size | 640 |
| Intermediate size | 2048 |
| Num layers | 18 |
| Num attention heads | 4 (GQA, 1 KV head) |
| Head dim | 256 |
| Vocab size | 262,144 |
| Max position embeddings | 1,048,576 (1M context) |
| Total parameters | 268,098,176 |
| VRAM (bfloat16) | 0.54 GB |
The attention stack uses grouped-query attention with one KV head. Five 512-token sliding-attention layers alternate with one full-attention layer. The repository configuration declares a 1,048,576-token maximum and extended RoPE parameters; that configuration value is not, by itself, evidence of reliable recall or generation across the full window.
LoRA Adapter Configuration (for episodic memory)
| Parameter | Value |
|---|---|
| Rank | 4 |
| Alpha | 8 |
| Target modules | gate_proj, up_proj, down_proj |
| Params per adapter | 580,608 |
| Per LoRA pair (one layer, one module) | 10,752 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("MonumentalSystems/Ouroboros-1MContext-Gemma-270m", torch_dtype="bfloat16")
tokenizer = AutoTokenizer.from_pretrained("MonumentalSystems/Ouroboros-1MContext-Gemma-270m")
Citation
Used in "Continuous Memory: Zero-Forgetting Episodic Memory via Per-Memory LoRA Adapters" (ICLR 2026 submission).
W&B runs: 41uzevnn, oa8xe89e, 2v01e4e9 (project: symbiogenesis)
License
This is a derivative of google/gemma-3-270m and is distributed under the
Gemma Terms of Use. It is not an Apache 2.0
weight release. Users are responsible for reviewing and complying with the Gemma
terms and any applicable restrictions.
- Task
- Text Generation
- Library
- transformers
- Type
- gemma3_text
- License
- gemma
- Languages
- en
- Created
- Mar 19, 2026
- Updated
- Oct 8, 2026