Llama-3.1-Instruct-abliterated-Nephilim_v3_via_adapter-8B

Llama-3.1-Instruct-abliterated-Nephilim_v3_via_adapter-8B

This repo contains a merge of pre-trained language models created using mergekit.

For this experiment, a LoRA extracted from a merge of models trained on Llama 3 8B Instruct was applied to Llama 3.1 8B Instruct. The theoretical goal was to obtain the stylistic benefit from the LoRA and the longer context of Llama 3.1. The resulting model is generally coherent. Tested at temperature=1 with minP=0.01.

Merge Details

Merge Method

This model was merged using the passthrough merge method using grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter + grimjim/Llama-3-Instruct-Nephilim-v3-LoRA-8B as a base.

Configuration

The following YAML configuration was used to produce this model:

base_model: grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter+grimjim/Llama-3-Instruct-Nephilim-v3-LoRA-8B
dtype: bfloat16
merge_method: passthrough
models:
  - model: grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter+grimjim/Llama-3-Instruct-Nephilim-v3-LoRA-8B