Downloads · 30 days
6
2% of all-time downloads
parrot-1_6B
maxim-saplin/parrot-1_6B
parrot-1_6B is a text generation model from maxim-saplin. Use it when you need the model to write or continue text. It is set up for transformers.
The model has been fine-tuned using LORA and trained to repeat user messages in ALL CAPS. It took just 11 minutes and 2 epochs (with 4k messages in each) to teach the base 1.6B Stable LM 2 model to follow chat structu…
All-time downloads
332
Public
Parameters
1.6B
3.3 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors3.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- stablelm
- Access
- Public
- Created
- Apr 20, 2024
- Updated
- Apr 23, 2024
- SHA
- c347aff5
Try a prompt
Model card
From the Hugging Face model README
The model has been fine-tuned using LORA and trained to repeat user messages in ALL CAPS. It took just 11 minutes and 2 epochs (with 4k messages in each) to teach the base 1.6B Stable LM 2 model to follow chat structure and learn the str.upper() behaviour. Trained on RTX 4060 8GB.
!!! Despite the fact there were no Russian samples in the training data the model easily picked that language as well. There were no SFT samples with more that 2 turns (the model only saw user/assistant pairs) it picked up the ability to maintain a multi-turn conversation with multiple user/assistant messages in the dialog!
Trainig code is here.
<p align="center"> <img src="https://github.com/maxim-saplin/parrot_sft/assets/7947027/b4eca263-c4fb-49f7-beb0-ce74f6f0b3e1" width="480"> </p>- Task
- Text Generation
- Library
- transformers
- Type
- stablelm
- Languages
- en, ru
- Created
- Apr 20, 2024
- Updated
- Apr 23, 2024