Downloads · 30 days
6
9% of all-time downloads
Mistral-7B-v0.1-1E-QLoRA-SFT-Test
Chahnwoo/Mistral-7B-v0.1-1E-QLoRA-SFT-Test
Mistral-7B-v0.1-1E-QLoRA-SFT-Test is a text generation model from Chahnwoo. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
MistralAI 7B model fine-tuned for 1 epoch on Dataricks instruction tuning dataset.
All-time downloads
70
Public
Parameters
7.2B
29 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors29 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Apr 18, 2024
- Updated
- Aug 1, 2024
- SHA
- 369a0af3
Try a prompt
Datasets
Model card
From the Hugging Face model README
Model Card for Model ID
MistralAI 7B model fine-tuned for 1 epoch on Dataricks instruction tuning dataset.
Model Details
Model Description
- Developed by: Andrew Chahnwoo Park
- Model type: Mistral
- Language(s) (NLP): English
- License: apache-2.0
- Finetuned from model: mistralai/Mistral-7B-v0.1
Mistral Repository
- Repository: mistralai/Mistral-7B-v0.1
Training Details
Training Data
Training Procedure
- Quantized Low-Rank Adaptation (QLoRA)
- Transformers Trainer
- DataCollatorForSeq2Seq
- Distributed Data Parallel (DDP) across two GPUs
Preprocessing
Manually created tokenized 'labels' for the dataset. Prompt template utilized basic template for instruction-tuning
Hardware
Performed fine-tuning with 2 * A100 GPUs
- Provided by Gnewsoft during work period Model and dataset are too large for free run sessions on Google Colab
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Languages
- en
- Created
- Apr 18, 2024
- Updated
- Aug 1, 2024