Downloads · 30 days
17
13% of all-time downloads
granite-3.1-3b-a800m-base-bnb-4bit
mdouglas/granite-3.1-3b-a800m-base-bnb-4bit
granite-3.1-3b-a800m-base-bnb-4bit is a text generation model from mdouglas. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
[!IMPORTANT] This repository is an experimental quantized version of the original model ibm-granite/granite-3.1-3b-a800m-base. It requires development versions of transformers and bitsandbytes.
All-time downloads
136
Public
Parameters
3.3B
2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2 GB · 100%
Parameter types
How the weights are stored.
U83.2B · 96%
Try a prompt
Base models
Model card
From the Hugging Face model README
[!IMPORTANT] This repository is an experimental quantized version of the original model
ibm-granite/granite-3.1-3b-a800m-base.It requires development versions of
transformersandbitsandbytes.
Quantization
The MLP expert parameters have been quantized in the NF4 format along with all nn.Linear modules except lm_head and router modules, using an experimental bnb_4bit_target_parameters configuration option.
Granite-3.1-3B-A800M-Base
Model Summary Granite-3.1-3B-A800M-Base is a decoder-only language model to support a variety of text-to-text generation tasks. It extends the context length of Granite-3.0-3B-A800M-Base from 4K to 128K
- Developers: Granite Team, IBM
- GitHub Repository: ibm-granite/granite-3.1-language-models
- Website: Granite Docs
- Paper: Granite 3.1 Language Models (coming soon)
- Release Date: December 18th, 2024
- License: Apache 2.0
Model Architecture: Granite-3.1-3B-A800M-Base is based on a decoder-only sparse Mixture of Experts (MoE) transformer architecture. Core components of this architecture are: Fine-grained Experts, Dropless Token Routing, and Load Balancing Loss.
| Model | 2B Dense | 8B Dense | 1B MoE | 3B MoE |
|---|---|---|---|---|
| Embedding size | 2048 | 4096 | 1024 | 1536 |
| Number of layers | 40 | 40 | 24 | 32 |
| Attention head size | 64 | 128 | 64 | 64 |
| Number of attention heads | 32 | 32 | 16 | 24 |
| Number of KV heads | 8 | 8 | 8 | 8 |
| MLP hidden size | 8192 | 12800 | 512 | 512 |
| MLP activation | SwiGLU | SwiGLU | SwiGLU | SwiGLU |
| Number of Experts | — | — | 32 | 40 |
| MoE TopK | — | — | 8 | 8 |
| Initialization std | 0.1 | 0.1 | 0.1 | 0.1 |
| Sequence Length | 4096 | 4096 | 4096 | 4096 |
| Position Embedding | RoPE | RoPE | RoPE | RoPE |
| # Parameters | 2.5B | 8.1B | 1.3B | 3.3B |
| # Active Parameters | 2.5B | 8.1B | 400M | 800M |
| # Training tokens | 12T | 12T | 10T | 10T |
- Task
- Text Generation
- Library
- transformers
- Type
- granitemoe
- License
- apache-2.0
- Languages
- en, de, es, fr, ja, pt
- Created
- Aug 2, 2025
- Updated
- Sep 18, 2025