PhelixZhen_-_Algae-550M-4bits

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Algae-550M - bnb 4bits

• Model creator: https://huggingface.co/PhelixZhen/
• Original model: https://huggingface.co/PhelixZhen/Algae-550M/

Original model description:

license: mit language: en

This is a pre-release model interface, training started on February 7, 2024, and the model will be released in the future.

The model adopts the Phi architecture, with 550 million parameters. It only supports English and does not support code writing.

The model's dataset is obtained by cleaning and deduplicating open-source datasets, with pre-training using approximately 30 billion instances.

If you are a native English speaker, you might find these sentences uncomfortable to read because the training of this model and the writing of this document were only completed by a very inexperienced Chinese high school student.

Anyway, this is a new attempt. It is trained on consumer-grade devices and without the guidance of professionals, so it's hard for us to expect it to perform exceptionally well.

But we hope this will be the beginning of a new great exploration.

(We have released a preview version on February 24, 2024, and you can run it using the following code：

from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
import torch

device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

tokenizer = AutoTokenizer.from_pretrained('pathtotokenizer')
model = AutoModelForCausalLM.from_pretrained('pathtomodel').to(device)
tokenizer.pad_token = tokenizer.eos_token

txt = 'inputtext'

# greedy search
gen_conf = GenerationConfig(
    num_beams=1,
    do_sample=True,
    max_length=700,
    no_repeat_ngram_size=6,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
    temperature=0.93,
    top_k=36,
    top_p=0.80
)

tokend = tokenizer.encode_plus(text=txt)
input_ids, attention_mask = torch.LongTensor([tokend.input_ids]).to(device), \
    torch.LongTensor([tokend.attention_mask]).to(device)

outputs = model.generate(
    inputs=input_ids,
    attention_mask=attention_mask,
    generation_config=gen_conf,
    
)

outs = tokenizer.decode(outputs[0].cpu().numpy(), clean_up_tokenization_spaces=True,)
print(outs)