Downloads · 30 days
0
rumi_fast
trung0209/rumi_fast
rumi_fast is a text generation model from trung0209. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Access
Public
Updated Mar 30, 2024
Repo size
101 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt67.2 MB · 48%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Access
- Public
- Created
- Mar 30, 2024
- Updated
- Mar 30, 2024
- SHA
- c6a99c20
Model card
From the Hugging Face model README
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_path = "PATH_TO_THIS_REPO"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
model_path,
device_map="auto",
torch_dtype='auto'
).eval()
# Prompt content: "hi"
messages = [
{"role": "user", "content": "hi"}
]
input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
output_ids = model.generate(input_ids.to('cuda'))
response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
# Model response: "Hello! How can I assist you today?"
print(response)
- Task
- Text Generation
- Library
- transformers
- License
- other
- Created
- Mar 30, 2024
- Updated
- Mar 30, 2024