Downloads · 30 days
7
14% of all-time downloads
JoshyG14/qwen3-1.7b-spanish-lora-no-inject
qwen3-1.7b-spanish-lora-no-inject is a text generation model from JoshyG14. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter for Qwen/Qwen3-1.7B, trained for Spanish sentence generation under morphosyntactic constraints without supplying the gold inflected form. This is Experiment B (LoRA-no-inject) from the MSc project: Gramma…
Downloads · 30 days
7
14% of all-time downloads
All-time downloads
50
Public
Repo size
69.8 MB
Likes
0
Public
Click a slice to open those files.
.safetensors69.8 MB · 100%
From the Hugging Face model README
LoRA adapter for Qwen/Qwen3-1.7B, trained for Spanish sentence generation under morphosyntactic constraints without supplying the gold inflected form. This is Experiment B (LoRA-no-inject) from the MSc project: Grammatically Controlled Sentence Generation with Small Language Models for Language Learning.
Code: github.com/JoshuaGraham14/LinguistOS.
Companion adapter trained with form injection: JoshyG14/qwen3-1.7b-spanish-lora-with-inject.
Given a Spanish verb lemma and morphosyntactic constraints (tense, person), generate a short natural sentence in which the correctly inflected form is the main verb.
The adapter is not a general-purpose Spanish chat model. Load it on top of Qwen/Qwen3-1.7B and use the same no-injection prompt format as at training time (see the repository configs).
| Base model | Qwen/Qwen3-1.7B (Apache 2.0) |
| Method | LoRA, rank (r=16), (\alpha=32), dropout 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Data | The same 4,205 filtered examples as LoRA-with-inject, with the injection line removed from every training prompt (3,995 train / 210 val; hard cells oversampled (\times 2)) |
| Objective | Supervised fine-tuning (next-token CE on prompt–completion pairs) |
| Checkpoint | Best validation loss at 1,000 steps (early stopping); 3 epochs, lr (2\times10^{-4}) |
| Adapter size | ~67 MB |
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_id = "Qwen/Qwen3-1.7B"
adapter_id = "JoshyG14/qwen3-1.7b-spanish-lora-no-inject"
tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(model, adapter_id)
PEFT 0.19.1 · TRL 1.8.0 · Transformers 5.13.0 · PyTorch 2.12.1