Downloads · 30 days
23
5% of all-time downloads
FWKV-50M
FWKV/FWKV-50M
FWKV-50M is a text generation model from FWKV. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
FWKV (Feed-forward Weighted Key Value, or Floored Weighted Key Value) is a novel efficient language model architecture proposed on May 13, 2026 by Me.
All-time downloads
479
Public
Parameters
50.6M
202 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors202 MB · 98%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- fwkv
- Access
- Public
- Created
- May 19, 2026
- Updated
- May 19, 2026
- SHA
- 7a485ded
Try a prompt
Datasets
Spaces
Model card
From the Hugging Face model README
FWKV
FWKV (Feed-forward Weighted Key Value, or Floored Weighted Key Value) is a novel efficient language model architecture proposed on May 13, 2026 by Me.
Design Philosophy
FWKV is built on two simple ideas:
- FFNN as the primary compute block — fast, parallelizable, hardware-friendly
- floor(W·K·V) as the sole context mechanism — a multiplicative recurrent state with a learned decay W floored above zero, so the model can never fully forget
Sample
Inference speed (CUDA -- T4)
Training
Factorised tied weights, chunked CE, bf16 training. Best val perplexity: 143.62
- Task
- Text Generation
- Library
- transformers
- Type
- fwkv
- License
- apache-2.0
- Created
- May 19, 2026
- Updated
- May 19, 2026