Downloads · 30 days
37
0% of all-time downloads
ShortKing-1.4b-v0.1
AtAndDev/ShortKing-1.4b-v0.1
ShortKing-1.4b-v0.1 is a text generation model from AtAndDev. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Model license: cc-by-nc-4.0<br This model is trained based on EleutherAI/pythia-1.4b-deduped model that is LoRA finetuned on vicgalle/alpaca-gpt4 dataset.<br
All-time downloads
73.2K
Public
Parameters
1.4B
5.7 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin2.8 GB · 50%
Parameter types
How the weights are stored.
F161.4B · 100%
Try a prompt
Datasets
Model card
From the Hugging Face model README
Model Overview
Model license: cc-by-nc-4.0<br> This model is trained based on EleutherAI/pythia-1.4b-deduped model that is LoRA finetuned on vicgalle/alpaca-gpt4 dataset.<br>
Prompt Template: Alpaca
<system_prompt>
### Instruction:
<user_message>
### Response:
<assistant_response>
Intended Use
THIS IS A TEST MODEL, IT IS NOT INTENDED FOR REAL APPLICATIONS BY ANY MEANS. HOWEVER, A NEW MODEL IS COMING IN THE SAME TOPIC.<br> This model series will be used for small but intense applications.
Training Details
This model took 2:31:23 to train in QLoRA on a single T4 GPU.<br>
- epochs:
1 - train batch size:
12 - eval batch size:
12 - gradient accumulation steps:
1 - maximum gradient normal:
0.3 - learning rate:
2e-4 - weight decay:
0.001 - optimizer:
paged_adamw_32bit - learning rate schedule:
cosine - warmup ratio (linear):
0.03
- Task
- Text Generation
- Library
- transformers
- Type
- gpt_neox
- License
- cc-by-nc-4.0
- Languages
- en
- Created
- Sep 25, 2023
- Updated
- Sep 29, 2023