qwen2.5-coder-1.5b-code-translation

qwen2.5-coder-1.5b-code-translation

A fine-tuned version of Qwen/Qwen2.5-Coder-1.5B-Instruct for translating code between C++, Java, and Python.

Training

• Base model: Qwen/Qwen2.5-Coder-1.5B-Instruct
• Method: LoRA (Low-Rank Adaptation) via LLaMA-Factory
• Dataset: tkeskin/leetcode-solutions (instruct config) — directed C++/Java/Python translation pairs derived from LeetCode solutions
• Hardware: AMD MI210 (ROCm) / NVIDIA CUDA, flash_attn: sdpa
• LoRA target: all linear layers (lora_target: all)
• Precision: bf16

Evaluation

Evaluated with an execution-based translation benchmark: each held-out evaluation-config payload from tkeskin/leetcode-solutions is a directed source→target translation whose output is compiled and run against the problem's input/output pairs. The eval split is held out from training (no leakage). Metric is pass@1 (all test cases pass), n-weighted over 3,336 payloads.

pass@1 by language pair × difficulty (%):

The base model also redefined the harness-provided ListNode/TreeNode helper types on ~6% of problems (a compile error); this fine-tune does so on none, having learned the dataset's convention. Full methodology is in the llm-fine-tune repo (Stage 5).

Standard benchmarks and the specialization trade-off

Translation specialization has a measurable cost on standard benchmarks (base → this model):

• Held-out perplexity on the translation test set drops 1.29 → 1.07 — training fit the target distribution.
• General ability is preserved — MMLU is flat (0.512 → 0.514); lm-eval reasoning tasks unchanged within noise.
• Code generation from a natural-language spec regresses — HumanEval pass@1 (Python) 75% → 53%.

That regression is mostly output-format specialization, not lost ability. Trained on LeetCode solutions — which wrap every answer in class Solution { ... } with camelCase methods — the model now answers HumanEval-style prompts in that same idiom, e.g. emitting class Solution { bool hasCloseElements(...) } instead of the requested free has_close_elements(...). The logic is frequently correct but mismatches the benchmark's free-function contract (C++ class-wrapping rises from 0% in the base to 78% here). The model improved at its trained format and regressed on unfamiliar ones, while keeping general knowledge intact.

Intended use

Given source code in one of C++, Java, or Python, the model generates a translation into the target language, following the same logic and structure. The Qwen2.5-Coder base model includes code-specific pre-training across C, C++, Java, Python, and many other languages, giving it a stronger prior for code structure.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "tkeskin/qwen2.5-coder-1.5b-code-translation"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": "Translate the following C++ code to Python:\n\nint add(int a, int b) { return a + b; }"
    }
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))