Downloads · 30 days
1
3% of all-time downloads
emilcw/gpt-sw3-356m-is-saga-dpo-nosft-antihack
gpt-sw3-356m-is-saga-dpo-nosft-antihack is a text generation model from emilcw. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
Fine-tuned with SAGA (Syntax-Aware Grammar Alignment), a two-stage pipeline that trains language models to generate grammatically correct Icelandic text using reinforcement learning from a symbolic parser oracle (Grey…
Downloads · 30 days
1
3% of all-time downloads
All-time downloads
35
Public
Repo size
26.3 MB
Likes
0
Public
Click a slice to open those files.
.safetensors25.2 MB · 96%
From the Hugging Face model README
Fine-tuned with SAGA (Syntax-Aware Grammar Alignment), a two-stage pipeline that trains language models to generate grammatically correct Icelandic text using reinforcement learning from a symbolic parser oracle (Greynir (Icelandic constituency parser)).
This is a LoRA adapter. Load it on top of AI-Sweden-Models/gpt-sw3-356m.
No-SFT ablation: DPO trained directly from the raw pretrained model (no KL-SFT warm-start). Antihack settings: mattr_weight=0.2, mattr_window=20.
| Metric | Base | + Δ-DPO |
|---|---|---|
| Parse success | 0.0% | 0.0% |
| Parse score | 0.000 | 0.000 |
| PPL-Wiki | 0.0 | 0.0 |
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("AI-Sweden-Models/gpt-sw3-356m", torch_dtype="auto")
model = PeftModel.from_pretrained(base, "{repo}")
tokenizer = AutoTokenizer.from_pretrained("AI-Sweden-Models/gpt-sw3-356m")
prompt = "Íslenska er"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=60, temperature=0.8, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))
@article{fakhar2025saga,
title={SAGA: Syntax-Aware Grammar Alignment for Low-Resource Nordic Languages},
author={Fakhar, Hoda and others},
year={2025},
note={Under review}
}
Inherits the base model license (AI Sweden LLM License / LumiOpen).