Downloads · 30 days
22
11% of all-time downloads
meta-llama.Meta-Llama-3-70B-Instruct_W8A8_FP8
espressor/meta-llama.Meta-Llama-3-70B-Instruct_W8A8_FP8
meta-llama.Meta-Llama-3-70B-Instruct_W8A8_FP8 is a text generation model from espressor. Use it when you need the model to write or continue text. It is set up for transformers.
This is a compressed model using llmcompressor.
All-time downloads
193
Public
Parameters
70.6B
72.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors72.7 GB · 100%
Parameter types
How the weights are stored.
F8_E4M368.5B · 97%
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
meta-llama/Meta-Llama-3-70B-Instruct - W8A8_FP8 Compression
This is a compressed model using llmcompressor.
Compression Configuration
- Base Model: meta-llama/Meta-Llama-3-70B-Instruct
- Compression Scheme: W8A8_FP8
- Dataset: HuggingFaceH4/ultrachat_200k
- Dataset Split: train_sft
- Number of Samples: 512
- Preprocessor: chat
- Maximum Sequence Length: 8192
Sample Output
Prompt:
Could not generate output
Output:
CUDA error: unknown error
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
Evaluation
<TODO>- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Nov 28, 2024
- Updated
- Nov 28, 2024