Downloads · 30 days
51
0% of all-time downloads
blockchainlabs_tinyllama_fusion_LHK_yunkong_v2
alnrg2arg/blockchainlabs_tinyllama_fusion_LHK_yunkong_v2
blockchainlabs_tinyllama_fusion_LHK_yunkong_v2 is a text generation model from alnrg2arg. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This model is based on the fusion strategy offered by Fanqi Wan(https://github.com/fanqiwan/FuseLLM).
All-time downloads
11.5K
Public
Parameters
1.1B
2.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Feb 19, 2024
- Updated
- Feb 19, 2024
- SHA
- 374cfd08
Try a prompt
Model card
From the Hugging Face model README
This model is based on the fusion strategy offered by Fanqi Wan(https://github.com/fanqiwan/FuseLLM).
Three models are fused together. 10epochs
Base model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
Blending model 1: HanNayeoniee/LHK_DPO_v1
Blending model 2: yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
This model will be optimized by Laser and DPO later.
This project is to make the on-device sLM. We are doing experiments on the models.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Created
- Feb 19, 2024
- Updated
- Feb 19, 2024