Downloads · 30 days
209
4% of all-time downloads
RichardErkhov/realshyfox_-_sharded-Llama-3-8B-gguf
realshyfox_-_sharded-Llama-3-8B-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
209
4% of all-time downloads
All-time downloads
5.4K
Public
Repo size
92.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf108 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
sharded-Llama-3-8B - GGUF
| Name | Quant method | Size |
|---|---|---|
| sharded-Llama-3-8B.Q2_K.gguf | Q2_K | 2.96GB |
| sharded-Llama-3-8B.IQ3_XS.gguf | IQ3_XS | 3.28GB |
| sharded-Llama-3-8B.IQ3_S.gguf | IQ3_S | 3.43GB |
| sharded-Llama-3-8B.Q3_K_S.gguf | Q3_K_S | 3.41GB |
| sharded-Llama-3-8B.IQ3_M.gguf | IQ3_M | 3.52GB |
| sharded-Llama-3-8B.Q3_K.gguf | Q3_K | 3.74GB |
| sharded-Llama-3-8B.Q3_K_M.gguf | Q3_K_M | 3.74GB |
| sharded-Llama-3-8B.Q3_K_L.gguf | Q3_K_L | 4.03GB |
| sharded-Llama-3-8B.IQ4_XS.gguf | IQ4_XS | 4.18GB |
| sharded-Llama-3-8B.Q4_0.gguf | Q4_0 | 4.34GB |
| sharded-Llama-3-8B.IQ4_NL.gguf | IQ4_NL | 4.38GB |
| sharded-Llama-3-8B.Q4_K_S.gguf | Q4_K_S | 4.37GB |
| sharded-Llama-3-8B.Q4_K.gguf | Q4_K | 4.58GB |
| sharded-Llama-3-8B.Q4_K_M.gguf | Q4_K_M | 4.58GB |
| sharded-Llama-3-8B.Q4_1.gguf | Q4_1 | 4.78GB |
| sharded-Llama-3-8B.Q5_0.gguf | Q5_0 | 5.21GB |
| sharded-Llama-3-8B.Q5_K_S.gguf | Q5_K_S | 5.21GB |
| sharded-Llama-3-8B.Q5_K.gguf | Q5_K | 5.34GB |
| sharded-Llama-3-8B.Q5_K_M.gguf | Q5_K_M | 5.34GB |
| sharded-Llama-3-8B.Q5_1.gguf | Q5_1 | 5.65GB |
| sharded-Llama-3-8B.Q6_K.gguf | Q6_K | 6.14GB |
| sharded-Llama-3-8B.Q8_0.gguf | Q8_0 | 7.95GB |
Meta-Llama-3-8B is an advanced language model developed by Meta, part of the Llama 3 family, optimized for text generation and natural language understanding tasks. This model leverages transformer architecture and is available in pre-trained and instruction-tuned variants.
This is the model card for the Meta-Llama-3-8B, a part of the Llama 3 model family which includes models with 8 billion and 70 billion parameters. The model is pre-trained on a diverse dataset of publicly available text and is designed for both research and commercial use, particularly for applications requiring natural language understanding and generation.
The model can be used directly for text generation tasks, including but not limited to chatbots, content creation, and interactive applications.
When fine-tuned, the model can be adapted for specific tasks such as sentiment analysis, question answering, and more.
The model should not be used for generating harmful content, spam, impersonation, or any other unethical activities as outlined in Meta's usage policy.
Users should be aware of the model's limitations and biases. It is recommended to use the model responsibly and with caution, especially in sensitive applications.
from transformers import pipeline
import torch
model_id = "meta-llama/Meta-Llama-3-8B"
pipeline = pipeline(
"text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
)
print(pipeline("Hello, how are you today?"))
The model was trained on a mix of publicly available online data, totaling over 15 trillion tokens.
The model utilizes Grouped-Query Attention (GQA) for improved inference scalability and was trained using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF).
Testing was performed using standard natural language processing benchmarks.
Metrics include standard NLP benchmarks for language models, such as perplexity, accuracy, and other task-specific measures.
Llama 3 models have demonstrated strong performance on various benchmarks, showing competitive results with other large-scale language models.
Carbon emissions can be estimated using the Machine Learning Impact calculator.
Llama 3 sharded model for an easier inference and fine tuning process on lower to mid end processing systems.
Llama 3 is an auto-regressive transformer-based language model designed for text generation and understanding.
Training utilized a cluster of NVIDIA A100 GPUs.
The model is compatible with PyTorch and Hugging Face's transformers library.
BibTeX:
@article{meta2024llama3,
title={Meta Llama 3: An Open-Source Large Language Model},
author={Meta AI},
journal={Meta AI Blog},
year={2024},
url={https://ai.meta.com/blog/meta-llama-3/}
}
APA:
Meta AI. (2024). Meta Llama 3: An Open-Source Large Language Model. Meta AI Blog. Retrieved from https://ai.meta.com/blog/meta-llama-3/
For more details about Meta, visit the Meta Llama website.
realshyfox
For any questions or feedback, please contact the Meta AI team via their GitHub repository.