Downloads · 30 days
3
2% of all-time downloads
Omid-sar/fine-tuning-llama2-7b-qlora-french
fine-tuning-llama2-7b-qlora-french is a machine learning model from Omid-sar. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
The following bitsandbytes quantization config was used during training: - quantmethod: bitsandbytes - loadin8bit: False - loadin4bit: True - llmint8threshold: 6.0 - llmint8skipmodules: None - llmint8enablefp32cpuoffl…
Downloads · 30 days
3
2% of all-time downloads
All-time downloads
148
Public
Repo size
269 MB
Likes
1
Public
Click a slice to open those files.
.bin134 MB · 100%
From the Hugging Face model README
The following bitsandbytes quantization config was used during training:
Fine-tuning Llama-2-7b using QLoRA in French on Google Colab
The goal of this project is to adapt the Llama-2-7b model, which initially might not have proficiency in French, to understand and respond accurately to queries in the French language. This adaptation involves fine-tuning the model on a dataset of French novels, allowing it to comprehend the nuances, syntax, and semantics of the French language. By leveraging the PEFT library from the Hugging Face ecosystem and QLoRA for more memory-efficient fine-tuning on a single T4 GPU provided by Google Colab, we aim to create a chatbot that can effectively answer questions posed in French.
This project involves several steps including setting up the environment, loading the dataset and model, configuring QLoRA and training parameters, training the model, and finally testing and pushing the fine-tuned model to Hugging Face.
Google Colab with T4 GPU
Python libraries: trl, transformers, accelerate, peft, datasets, bitsandbytes, einops