Downloads · 30 days
18
8% of all-time downloads
DeepSeek-R1-Distill-QweNSA-1.5B
doubleblind/DeepSeek-R1-Distill-QweNSA-1.5B
DeepSeek-R1-Distill-QweNSA-1.5B is a machine learning model from doubleblind. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains remote code and weights for a Native Sparse Attention distillation of DeepSeek-R1-Distill-Qwen-1.5B, distilled on mathematical reasoning data. Our parameter naming scheme refers to the paramet…
All-time downloads
222
Public
Parameters
2.7B
11 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors11 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen2nsa
- Access
- Public
- Created
- May 23, 2025
- Updated
- May 23, 2025
- SHA
- 6d6161d1
Datasets
Model card
From the Hugging Face model README
Quick Start
This repository contains remote code and weights for a Native Sparse Attention distillation of DeepSeek-R1-Distill-Qwen-1.5B, distilled on mathematical reasoning data. Our parameter naming scheme refers to the parameter count of the teacher model
Installation
To use this model, please ensure the following dependencies are installed:
Install the required Native Sparse Attention library from our custom fork:
pip install git+https://github.com/fnite1604/native-sparse-attention-pytorch.git
Install standard dependencies:
pip install transformers torch ...
Note: We recommend using the latest stable of Pytorch (currently 2.7.0) with CUDA 12.6 and the latest available version of Transformers
Example Usage
A quick_start.py script is included to help you get started with inference:
python quick_start.py
This will load the model and generate text based on a predefined prompt ("What is 1 + 1?") using our Native Sparse Attention enabled reasoning model.
- Type
- qwen2nsa
- License
- apache-2.0
- Languages
- en
- Created
- May 23, 2025
- Updated
- May 23, 2025