Downloads · 30 days
26
8% of all-time downloads
openaccess-ai-collective/StableLManticore-7B
StableLManticore-7B is a text generation model from openaccess-ai-collective. Use it when you need the model to write or continue text. It is set up for transformers.
Yeah, don't use this. It was mostly an experiment if it's even plausible. Unfortunately StableLM has poor support for SFT with the huggingface trainer, so no things like flash attention, etc. Ed result is this is near…
Downloads · 30 days
26
8% of all-time downloads
All-time downloads
335
Public
Repo size
64.1 GB
Likes
1
Public
Click a slice to open those files.
.bin16 GB · 100%
From the Hugging Face model README
Yeah, don't use this. It was mostly an experiment if it's even plausible. Unfortunately StableLM has poor support for SFT with the huggingface trainer, so no things like flash attention, etc. Ed result is this is nearly impossible to train efficiently. Yes, it's plausible to try to train this with LoRA, but it's not very usable at all.
WandB: https://wandb.ai/wing-lian/stable-manticore-7b/runs/b1qqzf2s