Downloads · 30 days
18
31% of all-time downloads
RDson_-_Phi-3-mini-code-finetune-128k-instruct-v1-awq
RichardErkhov/RDson_-_Phi-3-mini-code-finetune-128k-instruct-v1-awq
RDson_-_Phi-3-mini-code-finetune-128k-instruct-v1-awq is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
58
Public
Parameters
3.8B
2.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.3 GB · 100%
Parameter types
How the weights are stored.
I323.6B · 95%
Model card
From the Hugging Face model README
Quantization made by Richard Erkhov.
Phi-3-mini-code-finetune-128k-instruct-v1 - AWQ
- Model creator: https://huggingface.co/RDson/
- Original model: https://huggingface.co/RDson/Phi-3-mini-code-finetune-128k-instruct-v1/
Original model description:
license: other license_name: phi-3 license_link: https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/raw/main/LICENSE datasets:
- m-a-p/CodeFeedback-Filtered-Instruction tags:
- phi
- phi-3
- '3'
- code
Finetune of microsoft/Phi-3-mini-128k-instruct on m-a-p/CodeFeedback-Filtered-Instruction for ~9-10h using a single 3090 24GB.
Due to limited resources and time, the training was only on half (0.5136) of the epoch.
train_loss: 0.43311
learning_rate=5e-5,
lr_scheduler_type="cosine",
max_length=1024,
max_prompt_length=512,
overwrite_output_dir=True,
beta=0.1,
gradient_accumulation_steps=8,
optim="adamw_torch",
num_train_epochs=1,
evaluation_strategy="steps",
eval_steps=0.2,
logging_steps=1,
warmup_steps=50,
fp16=True,
save_steps=50
- Type
- phi3
- Created
- Nov 27, 2024
- Updated
- Nov 27, 2024