Downloads · 30 days
26
4% of all-time downloads
sascha-frank-ai-research/tsft-rag-gemma-3-12b-it
tsft-rag-gemma-3-12b-it is a text generation model from sascha-frank-ai-research. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation
Downloads · 30 days
26
4% of all-time downloads
All-time downloads
700
Public
Parameters
12.2B
24.4 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors24.4 GB · 100%
From the Hugging Face model README
Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation
Part of the TSFT-RAG model family for Retrieval-Augmented Generation across Gemma, Qwen and Llama architectures.
This repository contains TSFT-RAG Gemma-3-12B-IT, a full-parameter fine-tuned derivative of Google's Gemma-3-12B-IT developed within the TSFT-RAG (Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation) research project.
The objective is to investigate how supervised full fine-tuning changes the behaviour of modern language models when used as Retrieval-Augmented Generation (RAG) systems.
| Model | Parameters | Status |
|---|---|---|
| Gemma-3-1B-IT | 1B | Released |
| Gemma-3-4B-IT | 4B | Released |
| Gemma-3-12B-IT | 12B | Released |
General-purpose instruction models are optimized for broad conversational abilities but not specifically for Retrieval-Augmented Generation.
The TSFT-RAG project investigates whether full supervised fine-tuning can specialize foundation models for:
without changing the underlying transformer architecture.
The 12B model represents the largest Gemma model currently released within the TSFT-RAG project.
Reliable Retrieval-Augmented Generation requires considerably more than fluent language generation.
An effective RAG model should
The TSFT-RAG project evaluates these properties using identical datasets, prompts and evaluation procedures across Gemma, Qwen and Llama model families.
| Property | Value |
|---|---|
| Base model | google/gemma-3-12b-it |
| Training implementation | unsloth/gemma-3-12b-it |
| Architecture | Gemma3ForConditionalGeneration |
| Parameters | ~12 Billion |
| Hidden Size | 3840 |
| Transformer Layers | 48 |
| Attention Heads | 16 |
| Context Window | 131,072 tokens |
| Training Length | 1,024 tokens |
| Precision | bfloat16 |
| Fine-tuning | Full Parameter |
The TSFT-RAG benchmark contains supervised examples covering
The identical benchmark is used for every TSFT-RAG model to enable direct scientific comparison.
| Metric | Base | TSFT-RAG | Improvement |
|---|---|---|---|
| Aggregate Score | 0.450 | 0.686 | +0.236 |
| Hard Negative QA | 0.162 | 0.829 | +0.667 |
| Grounded QA | 0.613 | 0.594 | -0.019 |
| Topic Extraction | 0.651 | 0.632 | -0.019 |
| Key Message | 0.605 | 0.580 | -0.026 |
| Main Arguments | 0.589 | 0.614 | +0.025 |
| JSON Validity | 0.872 | 0.987 | +0.115 |
| Citation Precision | 0.000 | 0.950 | +0.950 |
| Citation Recall | 0.000 | 0.950 | +0.950 |
Among the currently released Gemma models, the 12B variant achieves the strongest overall performance.
The most notable improvements are observed for
Unlike many task-specific fine-tuning approaches, the model preserves strong grounded reasoning while substantially improving rejection behaviour and structured output quality.
These results demonstrate that full-parameter fine-tuning can effectively specialize large language models for Retrieval-Augmented Generation without architectural modifications.
Recommended for
Not intended as a general-purpose conversational assistant.
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained(
"sascha-frank-ai-research/tsft-rag-gemma-3-12b-it"
)
model = AutoModelForCausalLM.from_pretrained(
"sascha-frank-ai-research/tsft-rag-gemma-3-12b-it"
)
Training was performed using
All relevant configuration files, tokenizer files, evaluation summaries and licensing information are included in this repository.
The model should always be used together with appropriate source attribution and human oversight.
It should not replace expert judgement in legal, medical or other safety-critical domains.
This repository contains a modified derivative of Google's Gemma model.
Please consult
for detailed licensing information.
This model was developed and evaluated as part of the following research:
Frank, S., & Singh, R. (2026). Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation: A Multi-Family Evaluation Across Open-Weight Language Models.
Zenodo. DOI: 10.5281/zenodo.21638352
If you use this model in academic work, please cite the publication above.
@misc{Frank2026TSFTRAG,
author = {Frank, Sascha and Singh, Rawel},
title = {Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation:
A Multi-Family Evaluation Across Open-Weight Language Models},
year = {2026},
month = jul,
version = {v1},
publisher = {Zenodo},
doi = {10.5281/zenodo.21638352},
url = {https://doi.org/10.5281/zenodo.21638352},
note = {Preprint}
}
The complete TSFT-RAG research project, including training scripts, evaluation pipeline, benchmark methodology and documentation, is available at:
Project repository https://github.com/frankmst/rag-task-specific-full-finetuning
TSFT-RAG model family https://huggingface.co/sascha-frank-ai-research
Sascha Frank
Independent AI Researcher
ORCID https://orcid.org/0000-0002-0588-0081
GitHub https://github.com/frankmst
Hugging Face https://huggingface.co/sascha-frank-ai-research