Downloads · 30 days
20
36% of all-time downloads
RichardErkhov/anakin87_-_gemma-2b-orpo-awq
anakin87_-_gemma-2b-orpo-awq is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
20
36% of all-time downloads
All-time downloads
56
Public
Parameters
2.5B
2.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.1 GB · 98%
How the weights are stored.
I322B · 79%
From the Hugging Face model README
Quantization made by Richard Erkhov.
gemma-2b-orpo - AWQ
license: other license_name: gemma-terms-of-use license_link: https://ai.google.dev/gemma/terms library_name: transformers base_model: google/gemma-2b tags:
<img src="./assets/gemma-2b-orpo.png" width="450"></img>
This is an ORPO fine-tune of google/gemma-2b with
alvarobartt/dpo-mix-7k-simplified.
⚡ Quantized version (GGUF): https://huggingface.co/anakin87/gemma-2b-orpo-GGUF
ORPO (Odds Ratio Preference Optimization) is a new training paradigm that combines the usually separated phases of SFT (Supervised Fine-Tuning) and Preference Alignment (usually performed with RLHF or simpler methods like DPO).
gemma-2b-orpo performs well for its size on Nous' benchmark suite.
(evaluation conducted using LLM AutoEval).
| Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
|---|---|---|---|---|---|
| anakin87/gemma-2b-orpo 📄 | 39.45 | 23.76 | 58.25 | 44.47 | 31.32 |
| mlabonne/Gemmalpaca-2B 📄 | 38.39 | 24.48 | 51.22 | 47.02 | 30.85 |
| google/gemma-2b-it 📄 | 36.1 | 23.76 | 43.6 | 47.64 | 29.41 |
| google/gemma-2b 📄 | 34.26 | 22.7 | 43.35 | 39.96 | 31.03 |
Detailed results can be found here.
By comparison, on the Open LLM Leaderboard, google/gemma-2b-it has an average of 42.75.
| Metric | Value |
|---|---|
| Avg. | 47.35 |
| AI2 Reasoning Challenge (25-Shot) | 49.15 |
| HellaSwag (10-Shot) | 73.72 |
| MMLU (5-Shot) | 38.52 |
| TruthfulQA (0-shot) | 44.53 |
| Winogrande (5-shot) | 64.33 |
| GSM8k (5-shot) | 13.87 |
alvarobartt/dpo-mix-7k-simplified
is a simplified version of argilla/dpo-mix-7k.
You can find more information in the dataset card.
The model is small, so it runs smoothly on Colab. It is also fine to load the model using quantization.
# pip install transformers accelerate
import torch
from transformers import pipeline
pipe = pipeline("text-generation", model="anakin87/gemma-2b-orpo", torch_dtype=torch.bfloat16, device_map="auto")
messages = [{"role": "user", "content": "Write a rap song on Vim vs VSCode."}]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False)
outputs = pipe(prompt, max_new_tokens=500, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
The model was trained using HF TRL. 📓 Training notebook