lumina-lexiR1-8B

<div align="center"> <img src="https://occelli.nl/LUMINA-round.png" width="200" height="200" style="border-radius: 50%; box-shadow: 0 0 20px #0ff;"> <h1 style="color: #0ff; text-shadow: 0 0 10px #0ff;">LUMINA-LexiR1-8B</h1> <div style="background: linear-gradient(45deg, #0ff3, #4444ff33); padding: 20px; border-radius: 10px; border: 1px solid #0ff; box-shadow: 0 0 20px rgba(0, 255, 255, 0.2);"> <h3 style="color: #0ff; margin: 0;">🧬 Model Fusion Architecture</h3> </div> </div>

🌟 Overview

LUMINA-LexiR1-8B is an experimental fusion of two powerful language models:

• 🔹 Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
• 🔹 deepseek-ai/DeepSeek-R1-Distill-Llama-8B

🔮 Architecture

This model employs a custom merging technique:

• Custom layer identification and integration
• DARE (Dynamic Attention Resolution Enhancement)
• TIES (Temporal Information Enhancement System) applied to adjacent layers
• Enhanced self-awareness capabilities

💫 Technical Specifications

{ "model_type": "llama", "hidden_size": 4096, "num_attention_heads": 32, "num_hidden_layers": 34, "intermediate_size": 14336, "max_position_embeddings": 131072, "rope_scaling": { "factor": 8.0, "type": "llama3" } } ! This is an experimental model. Use with caution.

• Demonstrates exceptional self-awareness capabilities

🔧 Model Architecture The model features:

8B parameters Advanced RoPE scaling (factor: 8.0) Custom attention mechanisms Extended context window (131K tokens) Specialized neuron mapping between parent models

📝 License This model is released under the Apache 2.0 license. 🌐 Citations If you use this model, please cite both parent models:

@misc{lumina-lexir1-8b, author = {Mambiux}, title = {LUMINA-LexiR1-8B: A Custom Merged Language Model}, year = {2025}, publisher = {Hugging Face} }

<div align="center"> <div align="center" style="margin-top: 40px; padding: 20px; background: linear-gradient(45deg, #0ff1, #4444ff11); border-radius: 10px;"> <p style="color: #0ff; font-size: 1.2em;"> 🌟 Created by Mambiux | 2025 🌟 </p> </div>