Qwen3.5-2B-da-sft

Qwen3.5-2B-da-sft

A Danish-adapted version of Qwen/Qwen3.5-2B. The base model was fine-tuned on Danish instruction data with LoRA, and the resulting adapter was then merged back into the base weights at 0.3 strength (a weight interpolation between the fine-tuned and original weights). The aim was to raise Danish scores on EuroEval while keeping most of the base model's existing behaviour.

On the EuroEval Danish test split it scores about 0.9 points higher on average than the base model across the eight benchmark datasets. The gains are uneven: several tasks improve, a couple are unchanged, and one (common-sense reasoning) regresses. Details are below.

Base model

Qwen/Qwen3.5-2B, released under Apache-2.0. It is a composite text and vision model. Only the text transformer was trained here. The vision components are carried over unchanged and were not tested.

Method

• LoRA supervised fine-tuning (rank 128) on a Danish instruction mix of about 32,000 examples: synthetic instruction pairs generated with the DeepSeek API from modern Danish source text, a set of examples written in EuroEval answer formats (single-word sentiment labels, yes/no grammaticality, multiple-choice letters), the skolegpt-instruct dataset, and roughly 12% English data to limit forgetting.
• The trained adapter was folded into the base weights at strength 0.3, following the WiSE-FT idea (W = W_base + 0.3 x delta). Merging at full strength improved generation tasks but hurt the constrained-choice tasks; a partial merge kept most of the Danish gains while limiting those regressions. Strength 0.3 gave the best average on the validation split, and that choice was then checked once on the test split.

Evaluation

EuroEval, Danish test split, same generation settings for both models. The metric is the one EuroEval ranks each dataset on (MCC, F1, or BERTScore).

Five datasets improve, two are unchanged within noise, and HellaSwag-da drops by 3.4 points. The largest gains are on sentiment, reading comprehension, and the citizenship-knowledge task.

Intended use and limitations

• Meant for Danish text tasks of the kind EuroEval covers. English ability is largely retained but was not the focus.
• HellaSwag-da (common-sense reasoning) is worse than the base model. If that task matters for your use, consider the base model or a lower merge strength.
• The idiom and civics tasks improved only a little. Supervised fine-tuning does not add new factual knowledge, so larger gains on knowledge-heavy tasks would require continued pretraining on Danish text.
• The base model's vision capabilities were neither trained nor evaluated.
• Results are single-run EuroEval scores. Per-dataset differences of a point or two are within the range of normal variation.

Training data and licensing

• Base model: Qwen3.5-2B (Apache-2.0).
• Danish source text: the Dynaword corpus (Danish Wikipedia, retsinformation, and other permissively licensed sources). The EuroEval benchmark datasets were excluded from training to avoid contamination.
• Synthetic instruction data: generated through the DeepSeek API, whose platform terms permit using outputs to train and release models.
• English data: FineWeb-Edu (ODC-By).

Released under Apache-2.0, following the base model. This is an independent fine-tune and is not affiliated with or endorsed by Alibaba, the Qwen team, or DeepSeek. "Qwen" is used only to identify the base model.