Llama-3-8B-MLP-gguf

Overview

GGUF merged with base model version of Secretmoon/LoRA-Llama-3-MLP LoRA adapter, LoRA Alpha=48. Secretmoon/LoRA-Llama-3-MLP is 8-bit LoRA adapter for the Llama-3-8B model, primarily designed to expand the model's knowledge of the MLP:FiM (My Little Pony: Friendship is Magic) universe. This adapter is ideal for generating fan fiction, role-playing scenarios, and other creative projects. The training data includes factual content from the Fandom wiki and canonical fan works that deeply explore the universe.

Base Model

The base model for this adapter is Sao10K/L3-8B-Stheno-v3.1, an excellent fine-tuned version of the original Llama-3-8B. It excels in story writing and role-playing without suffering from degradation due to overfitting.

Training Details

• Dataset:
  1. Cleaned copy of the MLP Fandom Wiki, excluding information about recent and side projects unrelated to MLP:FiM. (Alpaca)
  2. Approximately 100 specially selected fan stories from FiMFiction. (RAW text)
  3. Additional data to train the model as a personal assistant and enhance its sensitivity to user emotions. (Alpaca)
• Training Duration: 3 hours
• Hardware: 1 x NVIDIA RTX A6000 48GB
• PEFT Type: LoRA 8-bit
• Sequence Length: 6144
• Batch Size: 2
• Num Epochs: 3
• Optimizer: AdamW_BNB_8bit
• Learning Rate Scheduler: Cosine
• Learning Rate: 0.00033
• LoRA R: 256
• Sample Packing: True
• LoRA Target Linear: True

Recommendations for LoRA Alpha (If you merge LoRA to the model yourself)

• 16: Low influence
• 48: Suggested optimal value (recommended)
• 64: High influence, significantly impacting model behavior
• 128: Very high influence, drastically changing language model behavior (not recommended)

How to Use

• llama.cpp The opensource framework for running GGUF LLM models on which all other interfaces are made.
• koboldcpp Lightweight open source fork llama.cpp with a simple graphical interface and many additional features. Optimized for RP.
• LM studio Proprietary free fork llama.cpp with a graphical interface.

Other:

<br> You can contact me on telegram @monstor86 or discord @starlight2288 <br> Also you can try some RP with this adapter for free in my bot on telegram @Luna_Pony_bot <img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>