no-prompt-1.3b

appvoid/no-prompt-1.3b

no-prompt-1.3b is a text generation model from appvoid. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

This model uses an 1.3 billion parameters model as base to be further fine-tuned on the same data as palmer. It works pretty good and even surpasses sota model on hellaswag.

Downloads · 30 days

4% of all-time downloads

All-time downloads

426

Public

Repo size

10.8 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin5.4 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: apache-2.0
Model type: llama
Access: Public
Created: Dec 5, 2023
Updated: Dec 15, 2023
SHA: 49acb3de

Try a prompt

Datasets

appvoid/no-prompt-15k

Model card

From the Hugging Face model README

palmer

no-prompt

a sheared-llama-1.3b fine-tuning

This model uses an 1.3 billion parameters model as base to be further fine-tuned on the same data as palmer. It works pretty good and even surpasses sota model on hellaswag.

evaluation

Model	ARC_C	HellaSwag	PIQA	Winogrande
tinyllama-2t	0.2807	0.5463	0.7067	0.5683
palmer-001	0.2807	0.5524	0.7106	0.5896
sheared-1.3b	0.2910	0.5935	0.7339	0.5809
no-prompt-1.3b	0.3157	0.6022	0.7334	0.5864
falcon-rw-1b-instruct-openorca (sota)	0.3362	0.5997	0.7394	0.6148

This model was trained on less than 25% of the dataset yet achieves competitive performance to current sota on open llm leaderboard.

training

Training took ~5 P100 gpu hours. It was trained on 15,000 gpt-4 shuffled samples. no-prompt was fine-tuned using lower learning rates ensuring it keeps as much general knowledge as possible.

prompt

no prompt

limitations

Hallucinations are frequent, just as any transformer model this size.

Task: Text Generation
Library: transformers
Type: llama
License: apache-2.0
Languages: en
Created: Dec 5, 2023
Updated: Dec 15, 2023