Downloads · 30 days
28
14% of all-time downloads
RichardErkhov/erfanzar_-_PGT-1B-2EP-gguf
erfanzar_-_PGT-1B-2EP-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
28
14% of all-time downloads
All-time downloads
194
Public
Repo size
12.1 GB
Likes
0
Public
Click a slice to open those files.
.gguf14 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
PGT-1B-2EP - GGUF
| Name | Quant method | Size |
|---|---|---|
| PGT-1B-2EP.Q2_K.gguf | Q2_K | 0.39GB |
| PGT-1B-2EP.IQ3_XS.gguf | IQ3_XS | 0.44GB |
| PGT-1B-2EP.IQ3_S.gguf | IQ3_S | 0.45GB |
| PGT-1B-2EP.Q3_K_S.gguf | Q3_K_S | 0.45GB |
| PGT-1B-2EP.IQ3_M.gguf | IQ3_M | 0.48GB |
| PGT-1B-2EP.Q3_K.gguf | Q3_K | 0.51GB |
| PGT-1B-2EP.Q3_K_M.gguf | Q3_K_M | 0.51GB |
| PGT-1B-2EP.Q3_K_L.gguf | Q3_K_L | 0.55GB |
| PGT-1B-2EP.IQ4_XS.gguf | IQ4_XS | 0.54GB |
| PGT-1B-2EP.Q4_0.gguf | Q4_0 | 0.56GB |
| PGT-1B-2EP.IQ4_NL.gguf | IQ4_NL | 0.56GB |
| PGT-1B-2EP.Q4_K_S.gguf | Q4_K_S | 0.56GB |
| PGT-1B-2EP.Q4_K.gguf | Q4_K | 0.61GB |
| PGT-1B-2EP.Q4_K_M.gguf | Q4_K_M | 0.61GB |
| PGT-1B-2EP.Q4_1.gguf | Q4_1 | 0.61GB |
| PGT-1B-2EP.Q5_0.gguf | Q5_0 | 0.66GB |
| PGT-1B-2EP.Q5_K_S.gguf | Q5_K_S | 0.66GB |
| PGT-1B-2EP.Q5_K.gguf | Q5_K | 0.71GB |
| PGT-1B-2EP.Q5_K_M.gguf | Q5_K_M | 0.71GB |
| PGT-1B-2EP.Q5_1.gguf | Q5_1 | 0.72GB |
| PGT-1B-2EP.Q6_K.gguf | Q6_K | 0.78GB |
| PGT-1B-2EP.Q8_0.gguf | Q8_0 | 1.0GB |
license: apache-2.0 datasets:
OST-OpenSourceTransformers Github
this model is only 1B but you can call it somehow an SOTA
this model can also run on 4 GB GPU RAM and know dialogs as well
from transformers import AutoTokenizer, AutoModelForCausalLM
from IPython.display import clear_output
import textwrap
tokenizer = AutoTokenizer.from_pretrained("erfanzar/PGT-1B-2EP")
model = AutoModelForCausalLM.from_pretrained("erfanzar/PGT-1B-2EP",device_map='auto',load_in_8bit=True)
verify_text = lambda txt : '\n'.join([textwrap.fill(txt, width=110) for txt in txt.split('\n')])
def ppp(text:str):
"""
pre processing prompt
"""
return f"<|prompter|> {text} <|endoftext|><|assistant|>"
def generate(text,max_new_tokens:int=1024,use_ppp:bool=False,b_pair=False):
text = ppp(text) if use_ppp else text
for i in range(max_new_tokens):
enc = tokenizer(text,return_tensors='pt',add_special_tokens=False)
text_r = text
enc = model.generate(enc.input_ids,max_new_tokens=1,pad_token_id=0)
text = tokenizer.decode(enc[0],skip_special_tokens=False)
text = text[:-4]+tokenizer.eos_token if text[-4:] == '\n\n\n\n' else text
if text.endswith(tokenizer.eos_token) or text.endswith('\n\n\n\n\n'):
yield text[len(text_r):] if b_pair else text
break
else:
yield text[len(text_r):] if b_pair else text
for v in generate('what is a gpu',512,True):
clear_output(wait=True)
print(verify_text(v),end='')
The Pythia Suite is NOT intended for deployment. It is not in itself a product and cannot be used for human-facing interactions. For example, the model may generate harmful or offensive text...
and also remember that this model is not good enough for Persian, French, and Dutch at least for this version