Downloads · 30 days
21
0% of all-time downloads
orpo-lora-phi2
Amu/orpo-lora-phi2
orpo-lora-phi2 is a text generation model from Amu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a fine-tuned version of microsoft/phi-2 using trl on ultrafeedback dataset.
All-time downloads
5.4K
Public
Parameters
2.8B
5.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- phi
- Access
- Public
- Created
- Mar 31, 2024
- Updated
- Mar 31, 2024
- SHA
- 646be9d7
Try a prompt
Base models
Model card
From the Hugging Face model README
outputs
This model is a fine-tuned version of microsoft/phi-2 using trl on ultrafeedback dataset.
What's new
A test for ORPO: Monolithic Preference Optimization without Reference Model method using trl library.
How to reproduce
accelerate launch --config_file=/path/to/trl/examples/accelerate_configs/deepspeed_zero2.yaml \
--num_processes 8 \
/path/to/trl/scripts/orpo.py \
--model_name_or_path="microsoft/phi-2" \
--per_device_train_batch_size 1 \
--max_steps 8000 \
--learning_rate 8e-5 \
--gradient_accumulation_steps 1 \
--logging_steps 20 \
--eval_steps 2000 \
--output_dir="orpo-lora-phi2" \
--optim rmsprop \
--warmup_steps 150 \
--bf16 \
--logging_first_step \
--no_remove_unused_columns \
--use_peft \
--lora_r=16 \
--lora_alpha=16 \
--dataset HuggingFaceH4/ultrafeedback_binarized
- Task
- Text Generation
- Library
- transformers
- Type
- phi
- License
- apache-2.0
- Languages
- en
- Created
- Mar 31, 2024
- Updated
- Mar 31, 2024