qwen3-1.7b-summarization-dare-linear

merged

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Linear DARE merge method using vohuutridung/qwen3-1.7b as a base.

Models Merged

The following models were included in the merge:

• vohuutridung/qwen3-1.7b-summarization-cnn
• vohuutridung/qwen3-1.7b-summarization-mediasum
• vohuutridung/qwen3-1.7b-summarization-arxiv-full

Configuration

The following YAML configuration was used to produce this model:

models:  
  - model: vohuutridung/qwen3-1.7b-summarization-cnn  
    parameters:  
      weight:  
        - filter: "model.embed_tokens"  
          value: 0.0  
        - filter: "lm_head"  
          value: 0.0  
        - value: 0.6  
      density:  
        - filter: "model.embed_tokens"  
          value: 1.0  
        - filter: "lm_head"  
          value: 1.0  
        - value: 0.9 
  
  - model: vohuutridung/qwen3-1.7b-summarization-arxiv-full  
    parameters:  
      weight:  
        - filter: "model.embed_tokens"  
          value: 0.0  
        - filter: "lm_head"  
          value: 0.0  
        - value: 0.6  
      density:  
        - filter: "model.embed_tokens"  
          value: 1.0  
        - filter: "lm_head"  
          value: 1.0  
        - value: 0.9  
  
  - model: vohuutridung/qwen3-1.7b-summarization-mediasum  
    parameters:  
      weight:  
        - filter: "model.embed_tokens"  
          value: 0.0  
        - filter: "lm_head"  
          value: 0.0  
        - value: 0.6  
      density:  
        - filter: "model.embed_tokens"  
          value: 1.0  
        - filter: "lm_head"  
          value: 1.0  
        - value: 0.9

merge_method: dare_linear
base_model: vohuutridung/qwen3-1.7b
dtype: bfloat16
tokenizer_source: base