Downloads · 30 days
13
3% of all-time downloads
pszemraj/Yi-6B-200K-Llama-sharded
Yi-6B-200K-Llama-sharded is a text generation model from pszemraj. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
https://huggingface.co/chargoddard/Yi-6B-200K-Llama sharded & in bf16 (colab free GPU loadable)
Downloads · 30 days
13
3% of all-time downloads
All-time downloads
372
Public
Parameters
6.1B
12.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors12.1 GB · 100%
From the Hugging Face model README
https://huggingface.co/chargoddard/Yi-6B-200K-Llama sharded & in bf16 (colab free GPU loadable)
Please note the custom license from the original still applies 🌝
# !pip install -U -q transformers accelerate sentencepiece bitsandbytes
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
"pszemraj/Yi-6B-200K-Llama-sharded", use_fast=False
)
model = AutoModelForCausalLM.from_pretrained(
"pszemraj/Yi-6B-200K-Llama-sharded", load_in_4bit=True
)
prompt = "Custom non-commercial software licenses are just so fun, aren't they? The best thing about them is"
if torch.cuda.is_available():
torch.cuda.empty_cache()
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
tokens = model.generate(
**inputs,
max_new_tokens=192,
temperature=0.75,
top_p=0.9,
no_repeat_ngram_size=4,
do_sample=True,
renormalize_logits=True,
repetition_penalty=1.05,
)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))