MN-2407-DSK-V1.0.0-Zeroed-Interleaved-17.7B

PJMixers-Dev/MN-2407-DSK-V1.0.0-Zeroed-Interleaved-17.7B

MN-2407-DSK-V1.0.0-Zeroed-Interleaved-17.7B is a text generation model from PJMixers-Dev. Use it when you need the model to write or continue text. It is set up for transformers.

1.5x layer count. Needs to be trained. Not useful on it's own, don't bother quantizing this.

Downloads · 30 days

14% of all-time downloads

All-time downloads

Public

Parameters

17.7B

35.4 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors35.4 GB · 100%

At a glance

Task: Text Generation
Library: transformers
Model type: mistral
Access: Public
Created: Mar 18, 2025
Updated: Mar 18, 2025
SHA: f2f3d422

Try a prompt

Base models

PocketDoc/Dans-SakuraKaze-V1.0.0-12b

Model card

From the Hugging Face model README

PJMixers-Dev/MN-2407-DSK-V1.0.0-Zeroed-Interleaved-17.7B

1.5x layer count. Needs to be trained. Not useful on it's own, don't bother quantizing this.

merge_method: passthrough
dtype: bfloat16
slices:
# Original first 10 layers (L0-L9)
- sources:
  - layer_range: [0, 10]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L10
- sources:
  - layer_range: [10, 11]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L10
- sources:
  - layer_range: [10, 11]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L11
- sources:
  - layer_range: [11, 12]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L11
- sources:
  - layer_range: [11, 12]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L12
- sources:
  - layer_range: [12, 13]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L12
- sources:
  - layer_range: [12, 13]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L13
- sources:
  - layer_range: [13, 14]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L13
- sources:
  - layer_range: [13, 14]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L14
- sources:
  - layer_range: [14, 15]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L14
- sources:
  - layer_range: [14, 15]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L15
- sources:
  - layer_range: [15, 16]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L15
- sources:
  - layer_range: [15, 16]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L16
- sources:
  - layer_range: [16, 17]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L16
- sources:
  - layer_range: [16, 17]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L17
- sources:
  - layer_range: [17, 18]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L17
- sources:
  - layer_range: [17, 18]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L18
- sources:
  - layer_range: [18, 19]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L18
- sources:
  - layer_range: [18, 19]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# "Before Dupe" of L19
- sources:
  - layer_range: [19, 20]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0
# Original L19
- sources:
  - layer_range: [19, 20]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

# Original L20
- sources:
  - layer_range: [20, 21]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L20
- sources:
  - layer_range: [20, 21]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L21
- sources:
  - layer_range: [21, 22]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L21
- sources:
  - layer_range: [21, 22]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L22
- sources:
  - layer_range: [22, 23]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L22
- sources:
  - layer_range: [22, 23]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L23
- sources:
  - layer_range: [23, 24]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L23
- sources:
  - layer_range: [23, 24]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L24
- sources:
  - layer_range: [24, 25]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L24
- sources:
  - layer_range: [24, 25]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L25
- sources:
  - layer_range: [25, 26]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L25
- sources:
  - layer_range: [25, 26]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L26
- sources:
  - layer_range: [26, 27]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L26
- sources:
  - layer_range: [26, 27]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L27
- sources:
  - layer_range: [27, 28]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L27
- sources:
  - layer_range: [27, 28]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L28
- sources:
  - layer_range: [28, 29]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L28
- sources:
  - layer_range: [28, 29]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original L29
- sources:
  - layer_range: [29, 30]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
# "After Dupe" of L29
- sources:
  - layer_range: [29, 30]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b
    parameters:
      scale:
      - filter: o_proj
        value: 0.0
      - filter: down_proj
        value: 0.0
      - value: 1.0

# Original last 10 layers (L30-L39)
- sources:
  - layer_range: [30, 40]
    model: PocketDoc/Dans-SakuraKaze-V1.0.0-12b

Task: Text Generation
Library: transformers
Type: mistral
Created: Mar 18, 2025
Updated: Mar 18, 2025