VicUnlocked-alpaca-30b-exl3-2.5bpw

<div align="center">

VicUnlocked-alpaca-30b

_{<code>EXL3</code>  ·  <b>2.5 bpw</b>  ·  10.7 GB  ·  Llama  ·  60 layers}

<br/>

</div>

---

[!NOTE] An ExLlamaV3 build of Aeala/VicUnlocked-alpaca-30b at 2.5 bits per weight. See Quants for sibling repos at other bit‑widths.

Quants

<div align="center"> </div>

Inference

<table> <thead> <tr> <th align="left" width="32%">Loader</th> <th align="left">Use it for</th> </tr> </thead> <tbody> <tr> <td><a href="https://github.com/theroyallab/tabbyAPI"><b>TabbyAPI</b></a></td> <td>OpenAI&#8209;compatible HTTP server. Drop&#8209;in for OpenAI clients.</td> </tr> <tr> <td><a href="https://github.com/oobabooga/text-generation-webui"><b>text&#8209;generation&#8209;webui</b></a></td> <td>Local chat UI. Pick the <i>ExLlamaV3</i> loader from the model dropdown.</td> </tr> <tr> <td><a href="https://github.com/turboderp-org/exllamav3"><b>ExLlamaV3</b></a></td> <td>Direct Python API for embedding the model in your own code or pipeline.</td> </tr> </tbody> </table>

[!TIP] VRAM at 2.5 bpw: ~10.7 GB weights + ~2 GB context. Comfortable on a single 16 GB card; tight on 12 GB with reduced context.

Download

pip install -U huggingface_hub

hf download \
  blockblockblock/VicUnlocked-alpaca-30b-exl3-2.5bpw \
  --local-dir ./VicUnlocked-alpaca-30b-exl3-2.5bpw

<details> <summary><b>Quantization recipe</b> &nbsp;_{(advanced &mdash; embedded in <code>quantization_config.json</code>)}</summary> <br/>

Loaded automatically by every ExLlamaV3 loader; reproduced here for searchability.

</details>

License & use

[!IMPORTANT] Use and license follow the base model. Quantization adds no additional restrictions. Refer to the upstream repository for terms, citation, and safety documentation.

---

<div align="center"> _{<i>Quantized with <a href="https://github.com/honkware/blockquant"><b>BlockQuant</b></a> &nbsp;&middot;&nbsp; convention&nbsp;<code>{org}/{model}-exl3-{bpw}bpw</code></i>} </div>