Downloads · 30 days
4
1% of all-time downloads
Meta-Llama-3.1-70B-Instruct-AWQ-INT4-Dequantized-FP32
vultr/Meta-Llama-3.1-70B-Instruct-AWQ-INT4-Dequantized-FP32
Meta-Llama-3.1-70B-Instruct-AWQ-INT4-Dequantized-FP32 is a text generation model from vultr. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.1.
The vultr/Meta-Llama-3.1-70B-Instruct-AWQ-INT4-Dequantized-FP32 model is a quantized version Meta-Llama-3.1-70B-Instruct that was dequantized from HuggingFace's AWS Int4 model and requantized and optimized to run on A…
All-time downloads
272
Public
Parameters
70.6B
282 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors282 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama3.1
- Model type
- llama
- Access
- Public
- Created
- Jan 28, 2025
- Updated
- Jan 31, 2025
- SHA
- 738c806c
Try a prompt
Base models
Model card
From the Hugging Face model README
Model Information
The vultr/Meta-Llama-3.1-70B-Instruct-AWQ-INT4-Dequantized-FP32 model is a quantized version Meta-Llama-3.1-70B-Instruct that was dequantized from HuggingFace's AWS Int4 model and requantized and optimized to run on AMD GPUs. It is a drop-in replacement for hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4.
Throughput: 68.74 requests/s, 43994.71 total tokens/s, 8798.94 output tokens/s
Model Details
Model Description
- Developed by: Meta
- Model type: Quantized Large Language Model
- Language(s) (NLP): English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
- License: Llama 3.1
- Dequantized From: hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4
Compute Infrastructure
- Vultr
Hardware
- AMD MI300X
Software
- ROCm
Model Author
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama3.1
- Languages
- en, de, fr, it, pt, hi
- Created
- Jan 28, 2025
- Updated
- Jan 31, 2025