Downloads · 30 days
37
30% of all-time downloads
kabachuha/G4-MeroMero-v2-31B-Heretic-ARA-LoRA
G4-MeroMero-v2-31B-Heretic-ARA-LoRA is a image-text-to-text model from kabachuha. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Made with the new 4bit ARA LoRA (usearalora) technique at home.
Downloads · 30 days
37
30% of all-time downloads
All-time downloads
124
Public
Repo size
674 MB
Likes
3
Public
Click a slice to open those files.
.safetensors642 MB · 95%
From the Hugging Face model README
Made with the new 4bit ARA LoRA (use_ara_lora) technique at home.
Because I don't want to spam 60+ GB uploads on huggingface, this model (natively created as lora) is distributed as a lora file. You can merge it into the original model using PEFT or, if you are using llama.cpp, you can use the convenient GGUF file (convert lora to gguf script) which is located in the "quantized" section.
llama.cpp example command:
./llama-server <your current command> --lora-scaled "/media/kabachuha/doc/G4-MeroMero-v2-31B-Heretic-ARA-LoRA.gguf:1.0"
| Metric | This model | Original model (G4-MeroMero-v2-31B) |
|---|---|---|
| KL divergence | 0.0785 | 0 (by definition) |
| Refusals | 8/100 | 97/100 |
Lower refusals indicate fewer content restrictions, while lower KL divergence indicates more closeness to the original model's baseline. Higher refusals cause more rejections, objections, pushbacks, lecturing, censorship, softening and deflections.
| Parameter | Value |
|---|---|
| start_layer_index | 26 |
| end_layer_index | 52 |
| preserve_good_behavior_weight | 0.8949 |
| steer_bad_behavior_weight | 0.0025 |
| overcorrect_relative_weight | 0.9535 |
| neighbor_count | 15 |