Downloads · 30 days
71
3% of all-time downloads
RichardErkhov/cmarkea_-_bloomz-3b-dpo-chat-gguf
cmarkea_-_bloomz-3b-dpo-chat-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
71
3% of all-time downloads
All-time downloads
2.5K
Public
Repo size
44.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf51.8 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
bloomz-3b-dpo-chat - GGUF
| Name | Quant method | Size |
|---|---|---|
| bloomz-3b-dpo-chat.Q2_K.gguf | Q2_K | 1.52GB |
| bloomz-3b-dpo-chat.IQ3_XS.gguf | IQ3_XS | 1.68GB |
| bloomz-3b-dpo-chat.IQ3_S.gguf | IQ3_S | 1.71GB |
| bloomz-3b-dpo-chat.Q3_K_S.gguf | Q3_K_S | 1.71GB |
| bloomz-3b-dpo-chat.IQ3_M.gguf | IQ3_M | 1.81GB |
| bloomz-3b-dpo-chat.Q3_K.gguf | Q3_K | 1.9GB |
| bloomz-3b-dpo-chat.Q3_K_M.gguf | Q3_K_M | 1.9GB |
| bloomz-3b-dpo-chat.Q3_K_L.gguf | Q3_K_L | 2.02GB |
| bloomz-3b-dpo-chat.IQ4_XS.gguf | IQ4_XS | 2.0GB |
| bloomz-3b-dpo-chat.Q4_0.gguf | Q4_0 | 2.08GB |
| bloomz-3b-dpo-chat.IQ4_NL.gguf | IQ4_NL | 2.09GB |
| bloomz-3b-dpo-chat.Q4_K_S.gguf | Q4_K_S | 2.09GB |
| bloomz-3b-dpo-chat.Q4_K.gguf | Q4_K | 2.24GB |
| bloomz-3b-dpo-chat.Q4_K_M.gguf | Q4_K_M | 2.24GB |
| bloomz-3b-dpo-chat.Q4_1.gguf | Q4_1 | 2.25GB |
| bloomz-3b-dpo-chat.Q5_0.gguf | Q5_0 | 2.43GB |
| bloomz-3b-dpo-chat.Q5_K_S.gguf | Q5_K_S | 2.43GB |
| bloomz-3b-dpo-chat.Q5_K.gguf | Q5_K | 2.55GB |
| bloomz-3b-dpo-chat.Q5_K_M.gguf | Q5_K_M | 2.55GB |
| bloomz-3b-dpo-chat.Q5_1.gguf | Q5_1 | 2.6GB |
| bloomz-3b-dpo-chat.Q6_K.gguf | Q6_K | 2.8GB |
| bloomz-3b-dpo-chat.Q8_0.gguf | Q8_0 | 3.62GB |
license: bigscience-bloom-rail-1.0 datasets:
The bloomz-3b-dpo-chat is a conversational model fine-tuned using Direct Preference Optimization (DPO) from the base bloomz-3b-sft-chat model. This model aims to provide high-quality conversational abilities in both English and French, leveraging the pre-trained strengths of its SFT (Supervised Fine-Tuning) predecessor.
Parent Model: bloomz-3b-sft-chat
The bloomz-3b-dpo-chat model builds upon the solid foundation of the bloomz-3b-sft-chat, which is notable for its chatbot-specific pre-training and efficient tokenization strategy. The DPO fine-tuning process enhances the model's ability to generate more human-preferred responses in conversational contexts.
The model was initially trained on both French and English datasets, ensuring high efficiency and performance in these languages. Due to the DPO process and potential data type changes (from float16 to bfloat16), the model's multilingual capabilities might not be as robust as its SFT predecessor, but fine-tuning can help in restoring performance in other languages.
This model is suitable for chatbot applications, customer service automation, and other conversational AI systems where bilingual (French and English) support is essential.
The bloomz-3b-dpo-chat model was trained using the Anthropic/hh-rlhf dataset, which includes:
Human Preference Data:
Evaluation of the model was conducted using the PoLL (Pool of LLM) technique, assessing performance on 100 French questions with scores aggregated from six evaluations (two per evaluator). The evaluators included GPT-4o, Gemini-1.5-pro, and Claude3.5-sonnet.
Performance Scores (on a scale of 5):
| Model | Score | # params |
|---|---|---|
| gpt-4o | 4.13 | N/A |
| mistralai/Mixtral-8x7B-Instruct-v0.1 | 3.71 | 46.7b |
| gpt-3.5-turbo | 3.66 | 175b |
| mistralai/Mistral-7B-Instruct-v0.2 | 1.98 | 7.25b |
| cmarkea/bloomz-7b1-mt-sft-chat | 1.69 | 7.1b |
| cmarkea/bloomz-3b-dpo-chat | 1.68 | 3b |
| cmarkea/bloomz-3b-sft-chat | 1.51 | 3b |
| croissantllm/CroissantLLMChat-v0.1 | 1.19 | 1.3b |
| cmarkea/bloomz-560m-sft-chat | 1.04 | 0.56b |
| OpenLLM-France/Claire-Mistral-7B-0.1 | 0.38 | 7.25b |
The bloomz-3b-dpo-chat model demonstrates improved performance over its SFT counterpart, particularly in zero-shot contexts, making it a competitive choice for production environments.
To utilize the bloomz-3b-dpo-chat model, format the prompt for chatbot interactions as follows:
</s>[human prompt 1]<s>[bot answer 1]</s>[human prompt 2]<s>
Example code to load the model using HuggingFace's pipeline:
from transformers import pipeline
model = pipeline("text-generation", "cmarkea/bloomz-3b-dpo-chat")
result = model("</s>C'est quoi le deep learning ?<s>", max_new_tokens=512)
result
[{'generated_text': "</s>C'est quoi le deep learning ?<s>L'apprentissage
en profondeur est un sous-ensemble de l'apprentissage automatique qui
utilise des réseaux de neurones artificiels pour apprendre à partir de
données. Ces réseaux sont conçus pour reconnaître des modèles dans les
données et peuvent être utilisés pour des tâches telles que la
reconnaissance d'images, le traitement du langage naturel et la
reconnaissance vocale."}]
@online{DeBloomzChat,
AUTHOR = {Cyrile Delestre},
URL = {https://huggingface.co/cmarkea/bloomz-3b-dpo-chat},
YEAR = {2024},
KEYWORDS = {NLP ; Transformers ; LLM ; Bloomz},
}