Downloads · 30 days
8
10% of all-time downloads
Meta-Llama-3-70B-Instruct-GPTQ
OxxoCodes/Meta-Llama-3-70B-Instruct-GPTQ
Meta-Llama-3-70B-Instruct-GPTQ is a text generation model from OxxoCodes. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
All-time downloads
78
Public
Repo size
39.8 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors39.8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- llama
- Access
- Public
- Created
- Apr 19, 2024
- Updated
- Apr 19, 2024
- SHA
- 39d559cd
Try a prompt
Model card
From the Hugging Face model README
OxxoCodes/Meta-Llama-3-70B-Instruct-GPTQ
Built with Meta Llama 3
Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
Model Description
This is a 4-bit GPTQ quantized version of meta-llama/Meta-Llama-3-8B-Instruct.
This model was quantized using the following quantization config:
quantize_config = BaseQuantizeConfig(
bits=4,
group_size=128,
desc_act=False,
damp_percent=0.1,
)
To use this model, you need to install AutoGPTQ. For detailed installation instructions, please refer to the AutoGPTQ GitHub repository.
Example Usage
from auto_gptq import AutoGPTQForCausalLM
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-70B-Instruct")
model = AutoGPTQForCausalLM.from_quantized("OxxoCodes/Meta-Llama-3-70B-Instruct-GPTQ")
output = model.generate(**tokenizer("The capitol of France is", return_tensors="pt").to(model.device))[0]
print(tokenizer.decode(output))
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- other
- Created
- Apr 19, 2024
- Updated
- Apr 19, 2024