Downloads · 30 days
23
6% of all-time downloads
weeix/openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b-GGUF
openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b-GGUF is a text generation model from weeix. Use it when you need the model to write or continue text. The card lists the license as other.
GGUF conversion and quantization of iapp/openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b, an open-weight Thai legal LLM built on a NVIDIA Nemotron-3-Nano-Omni-30B-A3B-Reasoning base (Mamba2-Transformer hybrid MoE, 3…
Downloads · 30 days
23
6% of all-time downloads
All-time downloads
376
Public
Repo size
58.1 GB
Likes
0
Public
Click a slice to open those files.
.gguf58.1 GB · 100%
From the Hugging Face model README
GGUF conversion and quantization of iapp/openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b, an open-weight Thai legal LLM built on a NVIDIA Nemotron-3-Nano-Omni-30B-A3B-Reasoning base (Mamba2-Transformer hybrid MoE, 30B total / ~3B active parameters per token). The base model exposes exact statute and มาตรา (section) citations in a structured JSON contract, tuned for both closed-book recall and retrieval-augmented (RAG) legal question answering.
This repo provides quantized .gguf files for use with llama.cpp and compatible runtimes (Ollama, LM Studio, Jan, koboldcpp, etc). For the model card, benchmark results, training details, and usage examples, see the original repository.
| File | Quant | Size | Notes |
|---|---|---|---|
openthai2.0-legal-thaillm-nemotron-3-nano-30B-a3B-Q8_0.gguf | Q8_0 | ~31.3 GB | Near-lossless, closest to full precision |
openthai2.0-legal-thaillm-nemotron-3-nano-30B-a3B-Q4_K_M.gguf | Q4_K_M | ~22.8 GB | Smaller footprint, some quality loss vs Q8_0/BF16 |
Note the 30B MoE with ~3B active parameters per token means these files are larger than a dense 30B model but should run at speeds closer to a ~3B dense model, VRAM/RAM permitting.
Converted from the original safetensors with llama.cpp's convert_hf_to_gguf.py, then quantized with llama-quantize:
python convert_hf_to_gguf.py path\to\openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b\
llama-quantize.exe openthai2.0-legal-thaillm-nemotron-3-nano-30B-a3B-BF16.gguf openthai2.0-legal-thaillm-nemotron-3-nano-30B-a3B-Q8_0.gguf Q8_0 6
llama-quantize.exe openthai2.0-legal-thaillm-nemotron-3-nano-30B-a3B-BF16.gguf openthai2.0-legal-thaillm-nemotron-3-nano-30B-a3B-Q4_K_M.gguf Q4_K_M 6
llama-cli -m openthai2.0-legal-thaillm-nemotron-3-nano-30B-a3B-Q4_K_M.gguf -p "your prompt here" -n 512
or as an OpenAI-compatible server:
llama-server -m openthai2.0-legal-thaillm-nemotron-3-nano-30B-a3B-Q8_0.gguf --host 0.0.0.0 --port 8000 -c 32768
The model supports a served context window of 32,768 tokens. See the original model card for the chat template, recommended generation settings (temperature/top_p per use case), and JSON citation-answer prompting format — these apply unchanged to the GGUF builds.
Distributed under the same NVIDIA Open Model Agreement as the base model and the original OpenThai 2.0 Legal release. This is a format conversion and quantization only — no weights were retrained or modified beyond quantization. All credit for the underlying model goes to the OpenThai team (AIEAT / iApp Technology) and NVIDIA.
As with the source model: outputs are decision support, not legal advice, and citations should be verified against current law by a qualified professional before being relied upon.