Downloads · 30 days
26
29% of all-time downloads
cosmo-1b-qlora-pythontest
Lambent/cosmo-1b-qlora-pythontest
cosmo-1b-qlora-pythontest is a text generation model from Lambent. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
All-time downloads
89
Public
Repo size
7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin3.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Apr 15, 2024
- Updated
- Apr 22, 2024
- SHA
- 8844f2a9
Try a prompt
Model card
From the Hugging Face model README
Catastrophic forgetting test results:
Initial evaluation loss on 1k subset of HuggingFaceTB/cosmopedia-100k dataset was 1.102. 100 steps of LISA training reduced this to 1.049.
Comparison to control: cosmo-1b started out with 1.003 loss on (a different subset of) dataset, increasing to 1.024 at 100 steps.
Axolotl config: Same as qdora version but without dora.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Created
- Apr 15, 2024
- Updated
- Apr 22, 2024