Skip to content

adamo1139

Hermes-3-Llama-3.1-8B_W8A8

adamo1139/Hermes-3-Llama-3.1-8B_W8A8

Hermes-3-Llama-3.1-8B_W8A8 is a machine learning model from adamo1139. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.1.

Amazingly quick to inference on Ampere GPUs like 3090 Ti. in INT8. In VLLM I left it on a task for 10 minutes with prompt caching, average fixed input around 2000, variable input around 200 and output around 200. Aver…

Downloads · 30 days

8

24% of all-time downloads

All-time downloads

34

Public

Parameters

8B

11.2 GB on disk

Likes

0

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors11.2 GB · 100%

Parameter types

How the weights are stored.

I87B · 87%

Type
llama
License
llama3.1
Created
Oct 14, 2024
Updated
Oct 15, 2024