Downloads · 30 days
18
5% of all-time downloads
Instruct_Phi2_Dolly15K
HenryJJ/Instruct_Phi2_Dolly15K
Instruct_Phi2_Dolly15K is a text generation model from HenryJJ. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Fine-tuned from phi2，used Dolly15k for the dataset. 90% for training, 10% validation. Trained for 2.0 epochs using QLora. Trained with 1024 context window.
All-time downloads
375
Public
Parameters
2.8B
5.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- phi-msft
- Access
- Public
- Created
- Jan 10, 2024
- Updated
- Jan 10, 2024
- SHA
- 943e911b
Try a prompt
Datasets
Model card
From the Hugging Face model README
Instruct_Phi2_Dolly15K
Fine-tuned from phi2，used Dolly15k for the dataset. 90% for training, 10% validation. Trained for 2.0 epochs using QLora. Trained with 1024 context window.
Model Details
- Trained by: trained by HenryJJ.
- Model type: Instruct_Phi2_Dolly15K is an auto-regressive language model based on the phi 2 transformer architecture.
- Language(s): English
- License for HenryJJ/Instruct_Phi2_Dolly15K: apache-2.0 license
Prompting
Prompt Template With Context
chatml format
<|im_start|>system
{instruction}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
Prompt Template Without Context
<|im_start|>system
{instruction}<|im_end|>
<|im_start|>assistant
Training script:
Fully opensourced at: https://github.com/hengjiUSTC/learn-llm/blob/main/trl_finetune.py. Run on 1 A10G instance for 4 hours.
python3 trl_finetune.py --config configs/phi2-dolly.yml
- Task
- Text Generation
- Library
- transformers
- Type
- phi-msft
- License
- apache-2.0
- Created
- Jan 10, 2024
- Updated
- Jan 10, 2024