IceAbsintheLoucheRP-7b

IceAbsintheLoucheRP-7b (Ice0.152-15.11-RP)

mistral v0.2 base

...wip...

Ice0.150-20.10-RP-ABLv3 Abliteration Configuration

Ice0.150-20.10-RP-ABLv3 is an abliterated version of IceAbsinthe-7b (Ice0.150-20.10-RP), created using grimjim's llm-abliteration tool, which can be found at https://github.com/jim-plus/llm-abliteration.

The process involves the following steps:

1. python measure.py -m H:\FModels\Ice0.150-20.10-RP -o H:\RModels\Ice0.150-20.10-RP-v2.ref -q 4bit --projected
2. python sharded_ablate.py H:\RModels\Ice0.150-20.10-RP-ABL-v3.yml

For better results with other models, it is recommended to add the --clip 0.999 flag during the measuring step. The YAML file needs to be configured for each specific model based on the graphs generated by analyze.py. For further details, please refer to the GitHub page.

Ice0.150-20.10-RP-ABL-v3.yml

model: H:\FModels\Ice0.150-20.10-RP
measurements: H:\RModels\Ice0.150-20.10-RP-v2.ref
output: H:\FModels\Ice0.150-20.10-RP-ABLv3
ablate:
  - layer: 12
    measurement: 17
    scale: 1.0
    sparsity: 0.00
  - layer: 13
    measurement: 17
    scale: 1.0
    sparsity: 0.00
  - layer: 14
    measurement: 17
    scale: 1.0
    sparsity: 0.00
  - layer: 15
    measurement: 17
    scale: 1.0
    sparsity: 0.00
  - layer: 16
    measurement: 17
    scale: 1.0
    sparsity: 0.00
  - layer: 17
    measurement: 17
    scale: 1.0
    sparsity: 0.00
  - layer: 18
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 19
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 20
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 21
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 22
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 23
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 24
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 25
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 26
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 27
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 28
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 29
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 30
    measurement: 25
    scale: 1.0
    sparsity: 0.00
  - layer: 31
    measurement: 25
    scale: 1.0
    sparsity: 0.00

Merge Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: icefog72/Ice0.150-20.10-RP-ABLv3
        layer_range: [0, 32]
      - model: icefog72/IceAbsinthe-7b
        layer_range: [0, 32]

merge_method: slerp
base_model: icefog72/IceAbsinthe-7b
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
dtype: bfloat16