Downloads · 30 days
17
3% of all-time downloads
d-Qwen2-0.5B
aloobun/d-Qwen2-0.5B
d-Qwen2-0.5B is a text generation model from aloobun. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
- This is a distillation experiment with Qwen2-1.5B as teacher and Qwen2-0.5B as student model respectively. - Samples were taken from the Pile dataset. - optimizer: SM3, scheduler: cosine with warmup, lr=2e-5
All-time downloads
489
Public
Parameters
494M
988 MB on disk
Likes
5
Public
Repo makeup
Click a slice to open those files.
.safetensors988 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen2
- Access
- Public
- Created
- Jun 7, 2024
- Updated
- Jun 7, 2024
- SHA
- 97c54e07
Try a prompt
Datasets
Model card
From the Hugging Face model README
- This is a distillation experiment with Qwen2-1.5B as teacher and Qwen2-0.5B as student model respectively.
- Samples were taken from the Pile dataset.
- optimizer: SM3, scheduler: cosine with warmup, lr=2e-5
Qwen2 is the new series of Qwen large language models. For Qwen2, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters, including a Mixture-of-Experts model. This repo contains distilled 0.5B Qwen2 language model.
- Task
- Text Generation
- Library
- transformers
- Type
- qwen2
- License
- apache-2.0
- Created
- Jun 7, 2024
- Updated
- Jun 7, 2024