Downloads · 30 days
9
24% of all-time downloads
Llama-3-8B-instruct-awq
oxfrug/Llama-3-8B-instruct-awq
Llama-3-8B-instruct-awq is a machine learning model from oxfrug. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for vllm. The card lists the license as llama3.
4-bit AWQ (W4A16, asymmetric) of AI-Sweden-Models/Llama-3-8B-instruct.
All-time downloads
37
Public
Parameters
8B
5.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.7 GB · 100%
Parameter types
How the weights are stored.
I327B · 87%
Base models
Model card
From the Hugging Face model README
AI-Sweden Llama-3-8B-instruct — AWQ W4A16
4-bit AWQ (W4A16, asymmetric) of AI-Sweden-Models/Llama-3-8B-instruct.
Official Hub files are BF16 and GGUF. EXL2 lives at
oxfrug/Llama-3-8B-instruct-exl2.
This repo is the vLLM / TGI pack.
Load
# vLLM
from vllm import LLM
llm = LLM("oxfrug/Llama-3-8B-instruct-awq")
hf download oxfrug/Llama-3-8B-instruct-awq --local-dir ./Llama-3-8B-instruct-awq
Format is compressed-tensors from llmcompressor, not classic AutoAWQ GEMM files.
Convert
- Tool: llmcompressor
AWQModifier+QuantizationModifier(scheme="W4A16_ASYM") lm_headleft in higher precision- Calibration: 256 rows × 512 tokens from a Nordic/English wiki mix already used for EXL2 Swedish-heavy cal (
sv_heavy.rows.jsonl). Not the frozen eval suite. - Hardware: RTX 3090
License
Meta Llama 3 Community License. Keep NOTICE. Derivative of Meta Llama 3 via AI Sweden’s instruct tune.
Smoke (greedy)
| Prompt | Output |
|---|---|
Vad heter Sveriges huvudstad? Ett ord. | Huvudstaden i Sverige är Stockholm. |
Name the capital of Sweden in one word. | Huvudstaden i Sverige är Stockholm. |
Same Swedish-first habit as the BF16 parent. This is not a suite score.
Sibling
| Pack | Repo |
|---|---|
| EXL2 4.0–6.0 bpw | oxfrug/Llama-3-8B-instruct-exl2 |
| Eval suite (do not train) | oxfrug/nordic-instruct-eval-v2 |
- Library
- vllm
- Type
- llama
- License
- llama3
- Languages
- sv, da, no, en
- Created
- Aug 16, 2026
- Updated
- Aug 16, 2026