Downloads · 30 days
0
LLama-Deepseek-Sparse-Attention
andresnowak/LLama-Deepseek-Sparse-Attention
LLama-Deepseek-Sparse-Attention is a machine learning model from andresnowak. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repo contains two variants of the model. The only intended difference is the use of partial RoPE.
Access
Public
Updated Dec 2, 2025
Repo size
5.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.1 GB · 99%
At a glance
- Access
- Public
- Created
- Nov 21, 2025
- Updated
- Dec 2, 2025
- SHA
- 63ba7717
Model card
From the Hugging Face model README
LLama-Deepseek-Sparse-Attention
This repo contains two variants of the model. The only intended difference is the use of partial RoPE.
Variants
Models continually trained with 1B tokens on FW-edu
-
run-1089676
- Partial RoPE on the indexer: no (standard/full RoPE disabled for the “partial” behavior)
- Path:
run-1089676/ - Files:
model.safetensors,config.json(plus tokenizer files if included)
-
run-1089496
- Partial RoPE on the indexer: yes
- Path:
run-1089496/ - Files:
model.safetensors,config.json(plus tokenizer files if included)
license: mit
- Created
- Nov 21, 2025
- Updated
- Dec 2, 2025