Downloads · 30 days
21
0% of all-time downloads
ShortKing-3b-v0.2
AtAndDev/ShortKing-3b-v0.2
ShortKing-3b-v0.2 is a text generation model from AtAndDev. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-4.0.
This model is finetuned on a merged dataset of: oasst1-en, alpaca-cleaned and airoboros-2.1-no-code on a base model: Marx-3b-V2 - License: "Creative-Commons-Attribution-4.0"
All-time downloads
23K
Public
Parameters
3.4B
13.7 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin6.9 GB · 50%
Parameter types
How the weights are stored.
F163.4B · 100%
Try a prompt
Model card
From the Hugging Face model README
Model overview
This model is finetuned on a merged dataset of: oasst1-en, alpaca-cleaned and airoboros-2.1-no-code on a base model: Marx-3b-V2
- License: "
Creative-Commons-Attribution-4.0" - Language: "
en" - Size: "
3.43b params"
Prompt template
Prompt template:
### SYSTEM:
<system_prompt_here>
### HUMAN:
<prompter_message_here>
### INPUT:
<input_text_here>
### RESPONSE:
<leave_a_blank_line_here>
Note: If you dont have a system or input text, do not include the tokens in the prompt.
Training Details
This model took 2:40:54 to train in LoRA on a single A100 40gb GPU.<br>
- epochs:
1 - train batch size:
8 - eval batch size:
8 - gradient accumulation steps:
1 - maximum gradient normal:
0.3 - learning rate:
2e-4 - weight decay:
0.001 - optimizer:
paged_adamw_32bit - learning rate schedule:
cosine - warmup ratio (linear):
0.03
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- cc-by-4.0
- Languages
- en
- Created
- Oct 1, 2023
- Updated
- Oct 2, 2023