Downloads · 30 days
8
5% of all-time downloads
komixenon/GoToCompany-Llama-Sahabat-AI-v2-70B-IT-BNB4
GoToCompany-Llama-Sahabat-AI-v2-70B-IT-BNB4 is a text generation model from komixenon. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.1.
https://huggingface.co/GoToCompany/Llama-Sahabat-AI-v2-70B-IT
Downloads · 30 days
8
5% of all-time downloads
All-time downloads
158
Public
Parameters
72.7B
39.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors39.5 GB · 100%
How the weights are stored.
U870.6B · 97%
From the Hugging Face model README
https://huggingface.co/GoToCompany/Llama-Sahabat-AI-v2-70B-IT
While the full model weights require approximately 36 GB of VRAM to load, this does not include memory needed for the KV cache, which is essential during inference. KV cache usage depends heavily on your serving backend (vLLM, SGLang, Aphrodite) and your GPU architecture — particularly whether it supports FP8, FP16, or BF16.
For lower VRAM environments, you may consider enabling features such as:
These optimizations are backend-specific and require proper configuration. Please consult your inference engine’s documentation for details.