jhana-mistral-GGUF

jhana-mistral-GGUF

This model is a fine-tuned version of TheBloke/Mistral-7B-Instruct-v0.1-GPTQ specifically trained for generating guided meditations. The fine-tuning was conducted on the "jhana-guided-meditations-collection" dataset available on Hugging Face, utilizing the QLoRA fine-tuning approach.

Model description

The model utilizes the LlamaTokenizer and is quantized for efficient load and execution. It is intended for generating mindful meditation scripts by understanding and generating contextually relevant content. This version has been optimized for better performance and lower resource utilization during inference.

Intended uses & limitations

This model is intended for generating text related to guided meditations. It may not perform well on unrelated tasks or general-purpose language understanding due to its specialized training.

Training and evaluation data

The model was trained on the "jhana-guided-meditations-collection" dataset, which consists of various guided meditation scripts. The data was preprocessed and tokenized using the LlamaTokenizer.

Training procedure

Training hyperparameters

• Learning Rate: 0.0002
• Batch Size: 8 for training, 8 for evaluation
• Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
• Scheduler: Cosine learning rate scheduler
• Training Steps: 250
• Mixed Precision Training: Utilized Native AMP

Training results

Training resulted in a model capable of generating coherent and contextually relevant meditation scripts, improving upon the base model's capabilities in this specific domain.

Framework versions

• PEFT: 0.10.0
• Transformers: 4.40.0.dev0
• Pytorch: 2.2.2+cu121
• Datasets: 2.18.0
• Tokenizers: 0.15.2

Quantization with llama.cpp

The model was quantized to enhance its efficiency and reduce its size, making it more suitable for deployment in various environments, including those with limited resources. The quantization process was performed using llama.cpp, following the steps outlined by Maxime Labonne in Quantize Llama models with GGUF and llama.cpp.

The process involved:

• Cloning the llama.cpp repository and setting it up with the required dependencies.
• Downloading the model to be quantized.
• Using the llama.cpp/convert.py script to convert the model to fp16 format, followed by quantization, significantly reducing the model's size while retaining its performance capabilities.

The quantization resulted in a compressed model with a significant reduction in size from 13813.02 MB to 4892.99 MB, enhancing its loading and inference speeds without compromising on the generation quality.