Downloads · 30 days
35
7% of all-time downloads
Naypa/OpenThai-2.0-Legal-ThaiLLM-Nemotron-3-Nano-30B-A3B-IQ4_NL-GGUF
OpenThai-2.0-Legal-ThaiLLM-Nemotron-3-Nano-30B-A3B-IQ4_NL-GGUF is a text generation model from Naypa. Use it when you need the model to write or continue text. It is set up for llama-cpp. The card lists the license as apache-2.0.
🇹🇭 Thai Legal AI Model — IQ4NL Quantized GGUF for llama.cpp
Downloads · 30 days
35
7% of all-time downloads
All-time downloads
469
Public
Repo size
18.2 GB
Likes
1
Public
Click a slice to open those files.
.gguf18.2 GB · 100%
From the Hugging Face model README
🇹🇭 Thai Legal AI Model — IQ4_NL Quantized GGUF for llama.cpp
This repository contains the IQ4_NL quantized GGUF version of the OpenThai 2.0 Legal ThaiLLM, based on NVIDIA's Nemotron-3-Nano-30B-A3B architecture.
| Property | Value |
|---|---|
| Base Model | iapp/openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b |
| Architecture | Nemotron H (Mamba + Attention + MoE Hybrid) |
| Parameters | ~30B total, ~3.5B active per token |
| Quantization | IQ4_NL (Iterative Optimized 4-bit Non-Linear) |
| Bits-per-Weight | 4.61 BPW |
| File Size | 17 GB |
| Fallback Tensors | 5 of 401 (specialized SSM/Mamba layers) |
llama-quantizellama-cli -m "OpenThai 2.0 Legal ThaiLLM (Nemotron-3-Nano-30B-A3B)_IQ4_NL.gguf" \
-n 1024 \
-t 8 \
-p "คำถามทางกฎหมายของคุณ..."
llama-server -m "OpenThai 2.0 Legal ThaiLLM (Nemotron-3-Nano-30B-A3B)_IQ4_NL.gguf" \
--host 0.0.0.0 \
--port 8080 \
--ctx-size 4096
from llama_cpp import Llama
model = Llama.from_pretrained(
repo_id="Naypa/OpenThai-2.0-Legal-ThaiLLM-Nemotron-3-Nano-30B-A3B-IQ4_NL-GGUF",
filename="OpenThai 2.0 Legal ThaiLLM (Nemotron-3-Nano-30B-A3B)_IQ4_NL.gguf",
n_ctx=4096,
n_gpu_layers=-1, # Offload all layers to GPU
)
output = model("คำถาม: กฎหมายไทยเกี่ยวกับ...", max_tokens=512)
print(output["choices"][0]["text"])
| Component | Minimum | Recommended |
|---|---|---|
| VRAM | 16 GB | 24 GB (RTX 3090) |
| RAM | 32 GB | 64 GB |
| Storage | 20 GB free | 30 GB free |
This model inherits the license from the base model: Apache 2.0
Quantized and shared by @Naypa
If you find this model useful, please ⭐ star the original repo and consider citing the Nemotron architecture paper.