uvl-qwen2.5-coder-7b-dpo-lora

UVL feature-model generator — LoRA adapter (Qwen/Qwen2.5-Coder-7B-Instruct)

LoRA adapter that fine-tunes Qwen/Qwen2.5-Coder-7B-Instruct to generate UVL (Universal Variability Language) feature models from a domain description and structural constraints (feature count, cross-tree constraints, depth, UVL level).

Trained with fine-tunning-uvl on the good/bad generations of the UVL_LLM_Guided_Generation benchmark: a DPO pass on top of the SFT checkpoint, using the valid/broken pairs as chosen/rejected.

Status: experimental. In our runs the DPO pass regressed valid-UVL rate versus the SFT checkpoint (raw 68% → 32% in one B300 run), so the SFT model is the recommended checkpoint. This adapter is published for reproducibility and further DPO experiments, not as the best generator.

Results

The DPO checkpoint underperformed SFT on the 62-prompt held-out test set (flamapy validator). See the SFT repos below for the headline numbers (77.42% raw / 96.77% after dedup).

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen2.5-Coder-7B-Instruct"
tok = AutoTokenizer.from_pretrained("jagalindo/uvl-qwen2.5-coder-7b-dpo-lora")
model = AutoModelForCausalLM.from_pretrained(base, dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "jagalindo/uvl-qwen2.5-coder-7b-dpo-lora")

messages = [
    {"role": "system", "content": "You are an expert in Software Product Lines ... generate valid UVL ..."},
    {"role": "user",   "content": "## Your task\nGenerate a valid UVL feature model for ..."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=1536, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

For best results, sample a few candidates and keep the first that parses with flamapy (retry-until-valid).

Limitations

Small/medium model on a niche DSL; not every generation parses on the first try. Use retry-until-valid + the project's light post-processing (dedup feature names, drop dangling constraints, quote non-identifier namespaces) to reach the headline rate. As noted above, prefer the SFT checkpoint for actual generation.

Related models

Part of the fine-tunning-uvl project — three artifacts from the same training run, all based on Qwen/Qwen2.5-Coder-7B-Instruct:

• jagalindo/uvl-qwen2.5-coder-7b-dpo-lora — this adapter: DPO LoRA (experimental; regressed vs SFT)
• jagalindo/uvl-qwen2.5-coder-7b-sft-lora — SFT LoRA adapter (apply on the full-precision base; recommended checkpoint)
• jagalindo/uvl-qwen2.5-coder-7b-sft-merged — merged bf16 full model (CUDA-ready, transformers)
• jagalindo/uvl-qwen2.5-coder-7b-sft — fused MLX 4-bit model (Apple Silicon / mlx-lm)