Downloads · 30 days
5
13% of all-time downloads
Hodfa71/gpt-sw3-1b3-da-saga-delta-dpo
gpt-sw3-1b3-da-saga-delta-dpo is a text generation model from Hodfa71. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
Fine-tuned with SAGA (Syntax-Aware Grammar Alignment), a two-stage pipeline that trains language models to generate grammatically correct Danish text using reinforcement learning from a symbolic parser oracle (SpaCy (…
Downloads · 30 days
5
13% of all-time downloads
All-time downloads
40
Public
Repo size
51.4 MB
Likes
0
Public
Click a slice to open those files.
.safetensors50.4 MB · 98%
From the Hugging Face model README
Fine-tuned with SAGA (Syntax-Aware Grammar Alignment), a two-stage pipeline that trains language models to generate grammatically correct Danish text using reinforcement learning from a symbolic parser oracle (SpaCy (Danish dependency parser)).
This is a LoRA adapter. Load it on top of AI-Sweden-Models/gpt-sw3-1.3b.
SFT skipped (base PS 86.0% ≥ 80% threshold). Δ-DPO applied directly to base.
| Metric | Base | + Δ-DPO |
|---|---|---|
| Parse success | 86.0% | 98.5% |
| Parse score | 0.421 | 0.686 |
| PPL-Wiki | 16.0 | 18.0 |
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("AI-Sweden-Models/gpt-sw3-1.3b", torch_dtype="auto")
model = PeftModel.from_pretrained(base, "Hodfa71/gpt-sw3-1b3-da-saga-delta-dpo")
tokenizer = AutoTokenizer.from_pretrained("AI-Sweden-Models/gpt-sw3-1.3b")
prompt = "Dansk er"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=60, temperature=0.8, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))
@article{fakhar2025saga,
title={SAGA: Syntax-Aware Grammar Alignment for Low-Resource Nordic Languages},
author={Fakhar, Hoda and others},
year={2025},
note={Under review}
}
Inherits the base model license (AI Sweden LLM License / LumiOpen).