Downloads · 30 days
5
14% of all-time downloads
JoshyG14/qwen3-1.7b-spanish-lora-with-inject
qwen3-1.7b-spanish-lora-with-inject is a text generation model from JoshyG14. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter for Qwen/Qwen3-1.7B, trained for Spanish sentence generation under morphosyntactic constraints with the gold inflected form supplied in the prompt. This is Experiment A (LoRA-with-inject) from the MSc pro…
Downloads · 30 days
5
14% of all-time downloads
All-time downloads
37
Public
Repo size
69.8 MB
Likes
0
Public
Click a slice to open those files.
.safetensors69.8 MB · 100%
From the Hugging Face model README
LoRA adapter for Qwen/Qwen3-1.7B, trained for Spanish sentence generation under morphosyntactic constraints with the gold inflected form supplied in the prompt. This is Experiment A (LoRA-with-inject) from the MSc project: Grammatically Controlled Sentence Generation with Small Language Models for Language Learning.
Code: github.com/JoshuaGraham14/LinguistOS.
Companion adapter trained without form injection: JoshyG14/qwen3-1.7b-spanish-lora-no-inject.
Given a Spanish verb lemma, morphosyntactic constraints (tense, person), and the required surface form, generate a short natural sentence in which that form is the main verb.
The adapter is not a general-purpose Spanish chat model. Load it on top of Qwen/Qwen3-1.7B and use the same form-injection prompt format as at training time (see the repository configs).
| Base model | Qwen/Qwen3-1.7B (Apache 2.0) |
| Method | LoRA, rank (r=16), (\alpha=32), dropout 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Data | 4,205 filtered prompt–completion pairs from the 150-verb diagnostic grid (3,995 train / 210 val; hard cells oversampled (\times 2)) |
| Objective | Supervised fine-tuning (next-token CE on prompt–completion pairs) |
| Checkpoint | Best validation loss at 1,000 steps (early stopping); 3 epochs, lr (2\times10^{-4}) |
| Adapter size | ~67 MB |
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_id = "Qwen/Qwen3-1.7B"
adapter_id = "JoshyG14/qwen3-1.7b-spanish-lora-with-inject"
tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(model, adapter_id)
PEFT 0.19.1 · TRL 1.8.0 · Transformers 5.13.0 · PyTorch 2.12.1