Downloads · 30 days
0
VytautoDidziojoUniversitetas/LT-ABS-abstractive-Gemma3-12b
LT-ABS-abstractive-Gemma3-12b is a text generation model from VytautoDidziojoUniversitetas. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as openrail.
Adapter Name: LT-ABS-abstractive-Gemma3-12b
Downloads · 30 days
0
Access
Public
Updated May 21, 2026
Repo size
1.1 GB
Likes
0
Public
Click a slice to open those files.
.safetensors548 MB · 94%
From the Hugging Face model README
Adapter Name: LT-ABS-abstractive-Gemma3-12b
Base Model: google/gemma-3-12b-it
Architecture: Gemma3 Instruct
Task: Abstractive summaries generation
This adapter must be used for lithuanian abstractive sumamries generation using Hugging Face transformers and peft libraries.
Installing required Python libraries. Used: Python ≥ 3.10
pip install -r requirements.txt
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch
base_model_id = "google/gemma-3-12b-it"
adapter_id = "VytautoDidziojoUniversitetas/LT-ABS-abstractive-Gemma3-12b"
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
base_model_id,
quantization_config=bnb_config,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()
text = "Jūsų lietuviškas tekstas čia..."
prompt = f"Sukurkite trumpą santrauką šiam lietuviškam tekstui:\n\n{text}\n\nSantrauka:"
inputs = tokenizer(prompt, return_tensors="pt", max_length=2500, truncation=True).to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=512,
do_sample=False,
pad_token_id=tokenizer.eos_token_id,
)
generated = tokenizer.decode(outputs[0], skip_special_tokens=True)
summary = generated.split("Santrauka:")[-1].strip()
print(summary)
Fine-tuned on a Lithuanian Summarisation Corpus covering four domains:
qlora_settings:
r: 32
lora_alpha: 64
target_modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
lora_dropout: 0.05
task_type: "CAUSAL_LM"
training:
per_device_train_batch_size: 2
gradient_accumulation_steps: 4
learning_rate: 1e-4
warmup_ratio: 0.1
weight_decay: 0.01
num_train_epochs: 5
bf16: True
lr_scheduler_type: "cosine"
optim: "AdamW"
Environment: Hugging Face Transformers (v4.50.0)
Hardware: 1× NVIDIA RTX A6000 ADA
Evaluated on the test split of VytautoDidziojoUniversitetas/LT_Summarisation_Corpus. Reference summaries are extractive (summary_extract column).
| Rouge-1 | Rouge-2 | Rouge-L | BertScore Precision | BertScore Recall | BertScore F1 |
|---|---|---|---|---|---|
| 51.0727 | 34.7232 | 39.0811 | 88.6933 | 92.5333 | 90.5461 |
Please cite the model as:
Vytautas Magnus University and Vilnius University. 2026. Abstract Corpora for Artificial Intelligence, Abstractive model. Hugging Face. https://huggingface.co/VytautoDidziojoUniversitetas/LT-ABS-abstractive-Gemma3-12b
BibTeX:
@misc{lt_acai_abs_2026,
author = {{Vytautas Magnus University} and {Vilnius University}},
title = {Lithuanian Abstract Corpora Abstractive Model},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/VytautoDidziojoUniversitetas/LT-ABS-abstractive-Gemma3-12b},
note = {Developed under the New Generation Lithuania plan, project 02-101-K-0001.}
}
Funded by the European Union — NextGenerationEU. Naujos kartos Lietuva.
This project is funded by the Recovery and Resilience Facility under the Lithuanian "Naujos kartos Lietuva" plan and by the State Budget of the Republic of Lithuania.
Project: Santraukų tekstynai dirbtiniam intelektui — Project No. 02-101-K-0001.