Downloads · 30 days
17
5% of all-time downloads
sparsetral-16x7B-v1
serpdotai/sparsetral-16x7B-v1
sparsetral-16x7B-v1 is a text generation model from serpdotai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
All-time downloads
371
Public
Parameters
7.5B
15 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors15 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- sparsetral
- Access
- Public
- Created
- Jan 30, 2024
- Updated
- Jan 30, 2024
- SHA
- 6ba08dc5
Try a prompt
Datasets
Model card
From the Hugging Face model README
prompt format
### System:\n{system}\n### Human:\n{user}\n### Assistant:\n"
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("serpdotai/sparsetral-16x7B-v1", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("serpdotai/sparsetral-16x7B-v1", device_map="auto", trust_remote_code=True).eval()
inputs = tokenizer('### System:\n\n### Human:\nHow are you?\n### Assistant:\n', return_tensors='pt')
inputs = inputs.to(model.device)
pred = model.generate(**inputs)
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
# I am doing well, thank you.
- Task
- Text Generation
- Library
- transformers
- Type
- sparsetral
- License
- apache-2.0
- Created
- Jan 30, 2024
- Updated
- Jan 30, 2024