Downloads · 30 days
8
24% of all-time downloads
Granite-2b-8bits-Autoround-GPTQ-sym
Siddharth63/Granite-2b-8bits-Autoround-GPTQ-sym
Granite-2b-8bits-Autoround-GPTQ-sym is a machine learning model from Siddharth63. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
All-time downloads
34
Public
Parameters
2.5B
2.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.7 GB · 100%
Parameter types
How the weights are stored.
I322.4B · 96%
Model card
From the Hugging Face model README
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda"
model_path =
tokenizer = AutoTokenizer.from_pretrained(model_path)
# drop device_map if running on CPU
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()
# change input text as desired
input_text = "Where is the Thomas J. Watson Research Center located?"
# tokenize the text
input_tokens = tokenizer(input_text, return_tensors="pt").to(device)
# generate output tokens
output = model.generate(**input_tokens,
max_length=4000)
# decode output tokens into text
output = tokenizer.batch_decode(output)
# print output
print(output)
- Type
- granite
- License
- apache-2.0
- Created
- Apr 29, 2025
- Updated
- Apr 29, 2025