Downloads · 30 days
24
3% of all-time downloads
BananaMind-2-Micro-Preview
BananaMind-Model-Previewers/BananaMind-2-Micro-Preview
BananaMind-2-Micro-Preview is a text generation model from BananaMind-Model-Previewers. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This is the 100% training preview of BananaMind 2 Micro, a 2,933,193-parameter base causal language model. It is not an instruction-tuned chat model.
All-time downloads
774
Public
Parameters
3.2M
256 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors12.8 MB · 90%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- bananamind2_micro
- Access
- Public
- Created
- Aug 10, 2026
- Updated
- Aug 10, 2026
- SHA
- f45a0e94
Try a prompt
Model card
From the Hugging Face model README
BananaMind 2 Micro Preview - 100%
This is the 100% training preview of BananaMind 2 Micro, a 2,933,193-parameter base causal language model. It is not an instruction-tuned chat model.
Architecture
| Field | Value |
|---|---|
| Parameters | 2,933,193 |
| Layers | 9 |
| Hidden size | 128 |
| Intermediate size | 512 |
| Query/KV heads | 4 / 2 |
| Head dimension | 32 |
| Context | 4,096 |
| Vocabulary | 2,048, tied |
| Attention | GQA, pre-RoPE QK norm |
| MLP | SwiGLU |
| Refresh | detached attention signal, kernel-9 causal depthwise gate, embedding reinjection |
The learned refresh scalar is initialized to zero. During cached generation, the causal convolution history is carried alongside each layer's K/V cache.
Training
| Field | Value |
|---|---|
| Progress | 100% |
| Tokens seen | 74,998,349,824 |
| Target tokens | 75,000,000,000 |
| Hardware | 8 x NVIDIA RTX PRO 6000 Blackwell Server Edition |
| Matrix optimizer | Stock torch.optim.Muon |
| Muon LR | 0.05 |
| Embedding/1D optimizer | AdamW |
| AdamW LR | 0.004 |
| Precision | bfloat16 autocast |
| Token range | FineWeb-Edu | DCLM | Cosmopedia v2 | FinePhrase | FineMath | NPSet2 |
|---|---|---|---|---|---|---|
| 0.00B-15.00B | 55% | 30% | 6% | 4% | 4% | 1% |
| 15.00B-30.00B | 48% | 26% | 9% | 8% | 7% | 2% |
| 30.00B-48.75B | 42% | 20% | 12% | 12% | 11% | 3% |
| 48.75B-63.75B | 36% | 16% | 14% | 15% | 15% | 4% |
| 63.75B-75.00B | 30% | 12% | 15% | 18% | 21% | 4% |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "BananaMind-Model-Previewers/BananaMind-2-Micro-Preview"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
- Task
- Text Generation
- Library
- transformers
- Type
- bananamind2_micro
- License
- other
- Languages
- en
- Created
- Aug 10, 2026
- Updated
- Aug 10, 2026