CosmoAlpacaLisa-0.3-1b

Lambent/CosmoAlpacaLisa-0.3-1b

CosmoAlpacaLisa-0.3-1b is a text generation model from Lambent. Use it when you need the model to write or continue text. It is set up for transformers.

Figured the model must have learned the data a somewhat different way for all three training methods. Decided to test if merging them would give generalization benefits. I think it didn't hurt at least!

Downloads · 30 days

24% of all-time downloads

All-time downloads

Public

Parameters

1.7B

3.5 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors3.5 GB · 100%

At a glance

Task: Text Generation
Library: transformers
Model type: llama
Access: Public
Created: Apr 5, 2024
Updated: Apr 5, 2024
SHA: 93ba1863

Try a prompt

Base models

Model card

From the Hugging Face model README

Model	AGIEval	GPT4All	TruthfulQA	Bigbench	Average
CosmoAlpacaLisa-0.3-1b	23.79	51.61	40.25	29.97	36.41

Model	AGIEval	GPT4All	TruthfulQA	Bigbench	Average
CosmoAlpacaLight-1b	24.28	51.31	40.33	29.47	36.35

lisacosmo

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the DARE TIES merge method using HuggingFaceTB/cosmo-1b as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: Lambent/CosmoAlpacaLisa-1b
    parameters:
      density: 1.0
      weight: 1.0
  - model: Lambent/CosmoAlpacaLisa-0.2-1b
    parameters:
      density: 1.0
      weight: 1.0
  - model: Lambent/CosmoAlpacaLight-1b
    parameters:
      density: 1.0
      weight: 1.0
merge_method: dare_ties
base_model: HuggingFaceTB/cosmo-1b
parameters:
  normalize: true
  int8_mask: false
dtype: float16

Task: Text Generation
Library: transformers
Type: llama
Created: Apr 5, 2024
Updated: Apr 5, 2024