Downloads · 30 days
23
1% of all-time downloads
Phi-3-mini-code-finetune-128k-instruct-v1
RDson/Phi-3-mini-code-finetune-128k-instruct-v1
Phi-3-mini-code-finetune-128k-instruct-v1 is a text generation model from RDson. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Finetune of microsoft/Phi-3-mini-128k-instruct on m-a-p/CodeFeedback-Filtered-Instruction for ~9-10h using a single 3090 24GB.
All-time downloads
2.3K
Public
Parameters
3.8B
15.3 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors15.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- phi3
- Access
- Public
- Created
- Apr 25, 2024
- Updated
- Apr 25, 2024
- SHA
- 989936b3
Try a prompt
Model card
From the Hugging Face model README
Finetune of microsoft/Phi-3-mini-128k-instruct on m-a-p/CodeFeedback-Filtered-Instruction for ~9-10h using a single 3090 24GB.
Due to limited resources and time, the training was only on half (0.5136) of the epoch.
train_loss: 0.43311
learning_rate=5e-5,
lr_scheduler_type="cosine",
max_length=1024,
max_prompt_length=512,
overwrite_output_dir=True,
beta=0.1,
gradient_accumulation_steps=8,
optim="adamw_torch",
num_train_epochs=1,
evaluation_strategy="steps",
eval_steps=0.2,
logging_steps=1,
warmup_steps=50,
fp16=True,
save_steps=50
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- other
- Created
- Apr 25, 2024
- Updated
- Apr 25, 2024