Downloads · 30 days
1.4K
35% of all-time downloads
Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF
SARC-Taigi-LLM-12b-GGUF is a text generation model from Speech-AI-Research-Center. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
This repository contains GGUF format model files for SARC-Taigi-LLM-12b. This model is a specialized version of google/gemma-3-12b-it, fine-tuned by the Speech AI Research Center (SARC) using IMA's 'Taiwan Tongues' Ta…
Downloads · 30 days
1.4K
35% of all-time downloads
All-time downloads
4.2K
Public
Repo size
71.9 GB
Likes
1
Public
Click a slice to open those files.
.gguf36 GB · 100%
From the Hugging Face model README
This repository contains GGUF format model files for SARC-Taigi-LLM-12b. This model is a specialized version of google/gemma-3-12b-it, fine-tuned by the Speech AI Research Center (SARC) using IMA's 'Taiwan Tongues' Taigi Datasets and QLoRA.
The GGUF versions are optimized for inference on consumer-grade hardware (CPUs or GPUs with limited VRAM) via llama.cpp, Ollama, or other compatible backends.
For a 12b model, we recommend Q4_K_M for the best balance between speed and linguistic precision.
| File Name | Method | RAM/VRAM Required | Description |
|---|---|---|---|
SARC-Taigi-LLM-12b-Q8_0.gguf | Q8_0 | ~14-16 GB | Near-lossless quality. Recommended for academic research and high-precision tasks. |
SARC-Taigi-LLM-12b-Q6_K.gguf | Q6_K | ~12-14 GB | High precision for specialized linguistic research. |
SARC-Taigi-LLM-12b-Q4_K_M.gguf | Q4_K_M | ~9-11 GB | Highly Recommended. Optimal quality/performance ratio for 12GB GPUs. |
SARC-Taigi-LLM-12b-Q3_K_L.gguf | Q3_K_L | ~7-9 GB | Lightweight version for memory-constrained or 8GB VRAM devices. |
QA Examples
The model underwent a two-stage training process designed to build a robust linguistic foundation, followed by instruction alignment:
taigi-literature): A diverse corpus of classical and modern Taigi literary works.We evaluated the models using the << 2020 Grand Challenge, Talk to AI (科技大擂台,與AI對話)>> Final-Test Dataset, which consists of 1,000 multiple-choice reading comprehension questions. This serves as a benchmark for Taigi language understanding:
| Model | Accuracy | Note | |
|---|---|---|---|
| Stage | Gemma-3-12b-it | Gemma-3-27b-it | |
| Original | 0.80320 | 0.86214 | Baseline performance |
| After CPT | 0.88312 | 0.92296 | Knowledge internalization |
| After SFT | 0.89610 | 0.92582 | Instruction alignment |
# Ensure Ollama is installed and your Ollama SSH public key (typically found at ~/.ollama/id_ed25519.pub) is registered in your Hugging Face account.
ollama pull huggingface.co/Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF:Q4_K_M
<img src="Ollama-1.png" style="width: 75%; margin: 5px;">
ollama run huggingface.co/Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF:Q4_K_M
<img src="Ollama-2.png" style="width: 55%; margin: 5px;">
ollama ps
<img src="Ollama-3.png" style="width: 90%; margin: 5px;">
ollama run huggingface.co/Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF:Q4_K_M
<img src="Ollama-4.png" style="width: 85%; margin: 5px;">
<img src="Ollama-5.png" style="width: 45%; margin: 5px;">
# Ensure you are in the llama.cpp directory
cd /path/to/llama.cpp
# Run the Taigi model with optional optimization flags
./build/bin/llama-cli \
-hf Speech-AI-Research-Center/SARC-Taigi-LLM-12b-GGUF:Q4_K_M \
-p "<start_of_turn>user\n用台語共我紹介一下你自己。<end_of_turn>\n<start_of_turn>model\n" \
-n 512 \
-ngl 99 \
--temp 0.7
-n, --n-predict (Default: 128)
-ngl, --n-gpu-layers (Default: 0)
--temp (Default: 0.8)
While the current release is the result of CPT and SFT, this is only the beginning. Our multi-stage alignment strategy includes:
Learn how to perform this multi-stage fine-tuning (CPT + SFT) with our custom callbacks for Loss minimization and Gap stability on GitHub:
[GitHub: SARC-Taigi-LLM Training Pipeline]
If you find this project useful, please cite the IMA's Taiwan Tongues resource page and the Speech AI Research Center organization pages on Hugging Face and GitHub.
@misc{ima_taiwan_2026,
title = {IMA-Taiwan},
author = {Information Management Association of R.O.C. (IMA)},
year = {2026},
howpublished = {https://huggingface.co/IMA-Taiwan},
note = {Hugging Face organization page for Taiwan Tongues resources}
}
@misc{sarc_hf_2026,
title = {Speech-AI-Research-Center},
author = {Speech AI Research Center (SARC)},
year = {2026},
howpublished = {https://huggingface.co/Speech-AI-Research-Center},
note = {Hugging Face organization page for released Taigi model adapters}
}
@misc{sarctaigillm_repo_2026,
title = {Speech-AI-Research-Center},
author = {Speech AI Research Center (SARC)},
year = {2026},
howpublished = {https://github.com/Speech-AI-Research-Center},
note = {GitHub organization page for released Taigi-LLM training project}
}
This model is subject to the Gemma Terms of Use. By using this model, you agree to comply with Google’s licensing requirements.