Downloads · 30 days
92
5% of all-time downloads
adamo1139/Danube3-500M-4chan-archive-0709-GGUF
Danube3-500M-4chan-archive-0709-GGUF is a machine learning model from adamo1139. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Danube3 500M model finetuned on adamo1139/Fal7acy4chanarchiveShareGPT which is essentially 250M tokens of chat data from 4chan, organized in coherent threads, capturing various boards.
Downloads · 30 days
92
5% of all-time downloads
All-time downloads
2K
Public
Repo size
4 GB
Likes
0
Public
Click a slice to open those files.
.gguf4 GB · 100%
From the Hugging Face model README
Danube3 500M model finetuned on adamo1139/Fal7acy_4chan_archive_ShareGPT which is essentially 250M tokens of chat data from 4chan, organized in coherent threads, capturing various boards.
ChatML prompt format, use system prompts such as "A chat on 4chan board /3/", "A chat on 4chan board /biz/" etc, as this was trained in.
This is a very small 500M model, so it's not very smart.
Dataset doesn't have correctly formatted newspaces, so quoted content doesn't format correctly.
Instead of this:
>what did you say
I didn't say nothing
It will print
>what did you say I didn't say nothing
1 epoch, 16-bit LoRA, 8192 seq length, 256 rank, 256 alpha, rslora True, batch size 8, learning rate 0.00004, embedding learning rate 0.00001, target modules ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj","embed_tokens", "lm_head"], cosing learning scheduler.