Llama-3.1-8B-RainbowLight-EtherealMix

ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix

Overview

ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix is a powerful fusion of ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes and invisietch/EtherealRainbow-v0.3-8B, utilizing SLERP (Spherical Linear Interpolation) for optimal blending of embeddings. This merge enhances reasoning, contextual understanding, and creative language generation while retaining ethical alignment and responsiveness.

---

🔥 Merged Models

• ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes - A highly optimized instruction-tuned model, built for nuanced, long-form reasoning.
• invisietch/EtherealRainbow-v0.3-8B - A dynamic conversational model with expanded alignment and expressiveness.

---

⚙️ Merge Configuration

The following YAML configuration defines how these models were fused using SLERP:

# Merge configuration for ZeroXClem-Llama-3.1-8B-RainbowLight-EtherealMix using SLERP

name: ZeroXClem-Llama-3.1-8B-RainbowLight-EtherealMix
slices:
  - sources:
      - model: ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes
        layer_range: [0, 32]
      - model: invisietch/EtherealRainbow-v0.3-8B
        layer_range: [0, 32]
merge_method: slerp
base_model: invisietch/EtherealRainbow-v0.3-8B
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

Why SLERP?

• Maintains Model Integrity: Ensures a smooth transition between feature spaces of both models.
• Preserves Semantic Meaning: Avoids interpolation collapse, keeping token embeddings rich in structure.
• Balanced Performance: Retains the best qualities from both parent models.

---

🚀 Capabilities

🌟 Enhanced Features

• Supercharged Instruction Following – More intuitive and context-aware.
• Advanced Conversational Flow – Generates human-like responses with coherence.
• Creative and Expressive Writing – Ideal for storytelling, summarization, and content generation.
• Expanded Knowledge Base – Merging brings broader factual recall and conceptual understanding.
• Flexible Alignment – A balance of compliance and open-ended response generation.

---

📥 Usage Instructions

Transformers

You can use the model via Hugging Face's transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Sample inference
prompt = "What are the implications of artificial intelligence in the future of education?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Ollama

For local execution with Ollama:

ollama run hf.co/ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix

---

📌 Important Notes

• License: Governed by Meta's Llama 3.1 Community License.
• Alignment Considerations: Users are responsible for ethical and compliant use.
• System Tokens: Follows Llama 3.1 tokenization standards for inference stability.
• Quantization: Use FP16 for optimal performance, though Q8 quantized versions may be available.

---

💜 Special Thanks

Deep gratitude to:

• @invisietch for EtherealRainbow-v0.3-8B.
• Hugging Face & Open-Source AI Community for their incredible contributions. 🚀💖

---

🔗 Resources

• Hugging Face Model Page
• Open LLM Leaderboard
• MergeKit Repository

---

✨ Merged with precision. Optimized for excellence. Experience RainbowLight EtherealMix today! ✨

Open LLM Leaderboard Evaluation Results

Detailed results can be found here