Downloads · 30 days
38
5% of all-time downloads
simaai/Llama-3.1-Swallow-8B-Instruct-v0.5-GPTQ-Safetensors
Llama-3.1-Swallow-8B-Instruct-v0.5-GPTQ-Safetensors is a text generation model from simaai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Pre-LLiMa quantized Hugging Face checkpoint for Sima.ai compilation and evaluation. It was produced from tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5 at source revision b1f8317099a97e790ec872c1225ca155979b4816.
Downloads · 30 days
38
5% of all-time downloads
All-time downloads
729
Public
Parameters
8B
4.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors4.9 GB · 100%
How the weights are stored.
I327.5B · 93%
From the Hugging Face model README
Pre-LLiMa quantized Hugging Face checkpoint for Sima.ai compilation and
evaluation. It was produced from
tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.5 at source revision
b1f8317099a97e790ec872c1225ca155979b4816.
| Component | Method | Configuration |
|---|---|---|
224 decoder Linear modules | GPTQ | symmetric INT4, group size 256, static activation order |
lm_head | GPTQ | symmetric INT4, group size 256, static activation order |
All 225 Linear modules are quantized. There are no intentional BF16/FP16
language-layer exceptions. GPTQ used Hessian block size 128 and dampening
fraction 0.01. The block size is a GPTQ optimization parameter; the weight
quantization group size is 256.
The checkpoint uses the compressed-tensors packed Safetensors format.
Calibration used 512 deterministic, fixed-length 1,024-token samples with seed 0:
tokyotech-llm/swallow-magpie-ultra-v0.1HuggingFaceH4/ultrachat_200kDataset revisions and the complete recipe are pinned in recipe.yaml.
The included quantize.py is the exact model-specific quantization script used
to create this artifact. It accepts only the source and output paths:
source /project/davinci_users/software/florian.voss/Workspace/qwen_awq/venv/bin/activate
python quantize.py \
--model-path /project/mlasw/share/huggingface/models--tokyotech-llm--Llama-3.1-Swallow-8B-Instruct-v0.5 \
--output-dir /path/to/new-output
The artifact also includes recipe.yaml, versions.txt, model/tokenizer
configuration, the packed Safetensors weights, and the upstream license files.
This is a pre-LLiMa Hugging Face checkpoint intended for compilation and deployment evaluation on Sima.ai hardware. Keep compiler output separate from this checkpoint.
Quantization quality can vary by language, domain, prompt format, task, and context length. The 100-example screen is not a substitute for full evaluation of the intended deployment workload.
This checkpoint remains subject to the source model's Llama 3.3 and Gemma
license terms, intended-use guidance, and limitations. The corresponding
LICENSE, GEMMA_TERMS_OF_USE.md, USE_POLICY.md, and Notice files are
included.