Riemannian-Redshift-12B-v1

[!NOTE] <span style="color:red; font-weight:bold">⚠️ Note:</span> This model requires Mistral Tekken chat template.

🌌 Riemannian Redshift 12B v1

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This is an experimental karcher merge of several high quality Vortex5 models. I used float32 precision and max_iter: 1000 to ensure the best bits were chosen for the Riemannian center. This merge took 5 hours using graph_v18 as an accelerant with 8GB VRAM.

This model was merged using the Karcher Mean merge method.

Models Merged

The following models were included in the merge:

• Vortex5/Maroon-Sunset-12B
• Vortex5/Azure-Starlight-12B
• Vortex5/Scarlet-Seraph-12B
• Vortex5/Amber-Starlight-12B
• Vortex5/Shining-Seraph-12B
• Vortex5/Red-Synthesis-12B
• Vortex5/Starlit-Shadow-12B
• Vortex5/Crimson-Constellation-12B
• Vortex5/Vermilion-Sage-12B
• Vortex5/Astral-Noctra-12B

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: B:/12B/models--Vortex5--Astral-Noctra-12B
  - model: B:/12B/models--Vortex5--Azure-Starlight-12B
  - model: B:/12B/models--Vortex5--Crimson-Constellation-12B
  - model: B:/12B/models--Vortex5--Red-Synthesis-12B
  - model: B:/12B/models--Vortex5--Shining-Seraph-12B
  - model: B:/12B/models--Vortex5--Starlit-Shadow-12B
  - model: B:/12B/models--Vortex5--Vermilion-Sage-12B
  - model: B:/12B/models--Vortex5--Scarlet-Seraph-12B
  - model: B:/12B/models--Vortex5--Maroon-Sunset-12B
  - model: B:/12B/models--Vortex5--Amber-Starlight-12B
merge_method: karcher
parameters:
  max_iter: 1000
  tol: 1.0e-9
dtype: float32
out_dtype: bfloat16
tokenizer:  
  source: union
chat_template: auto
name: 🌌 Riemannian-Redshift-12B-v1