zephyr-qwen2.5-coder-1.5b-lora

---

base_model: Qwen/Qwen2.5-Coder-1.5B

library_name: peft

pipeline_tag: text-generation

tags:

- lora

- code-completion

- zephyr

- rtos

- c

- cpp

---

# Zephyr RTOS Code Completion — Qwen2.5-Coder-1.5B LoRA

This is a LoRA adapter for completing Zephyr RTOS source code. It was trained on a cleaned, leakage-controlled snapshot of the Zephyr repository.

## Base model

- Model: Qwen/Qwen2.5-Coder-1.5B

- Revision: df3ce67c0e24480f20468b6ef2894622d69eb73b

- Adapter type: LoRA

- LoRA rank: 16

- LoRA alpha: 32

- Training hardware: one NVIDIA TITAN RTX GPU

This release contains adapter weights, not a complete merged model. The original Qwen base model is required for inference.

## Training data

- Zephyr version: v4.4.1

- Zephyr commit: 1f6485eca25431b5ff27ce9a754218c9e559bbbb

- Cleaned source files: 12,823

- Original training examples: 64,935

- Validation examples: 4,774

- Evaluation examples: 4,517

- Languages: C, C headers, C++, and C++ headers

- Maximum sequence length: 1,024 tokens

Directory-group and exact-file-hash isolation were used to prevent source leakage across training, validation, and evaluation splits.

## Held-out evaluation

| Benchmark | Base NLL | LoRA NLL | NLL change | Accuracy change |

|---|---:|---:|---:|---:|

| General Zephyr, 512 examples | 0.950232 | 0.768027 | 19.17% reduction | +3.10 points |

| C++, 62 examples | 0.917017 | 1.059281 | 15.51% regression | -0.43 points |

General generation results:

- Exact-match improvement: +4.10 points

- First-line-match improvement: +5.27 points

- Edit-similarity improvement: +5.11 points

## Important limitation

This adapter provides the strongest general Zephyr improvement from the 1.5B experiments, but it regressed on the dedicated C++ benchmark. It should therefore be presented as the original single-GPU, general Zephyr adapter—not as the best C++ adapter.

The separately released 7B checkpoint provides positive held-out lift on both the general and dedicated C++ benchmarks.

## Usage

import torch

from peft import PeftModel

from transformers import AutoModelForCausalLM, AutoTokenizer



base\_model\_name = "Qwen/Qwen2.5-Coder-1.5B"

base\_revision = "df3ce67c0e24480f20468b6ef2894622d69eb73b"

adapter\_path = "."



tokenizer = AutoTokenizer.from\_pretrained(adapter\_path)



base\_model = AutoModelForCausalLM.from\_pretrained(

    base\_model\_name,

    revision=base\_revision,

    torch\_dtype=torch.float16,

    device\_map="auto",

)



model = PeftModel.from\_pretrained(base\_model, adapter\_path)

model.eval()



prompt = """#include <zephyr/kernel.h>



int main(void)

{

"""



inputs = tokenizer(prompt, return\_tensors="pt").to(model.device)



with torch.no\_grad():

&#x20;   output = model.generate(

&#x20;       \*\*inputs,

&#x20;       max\_new\_tokens=128,

&#x20;       do\_sample=False,

&#x20;   )



print(tokenizer.decode(output\[0], skip\_special\_tokens=True))

## Intended use

This adapter is intended for research and demonstration of parameter-efficient domain adaptation for Zephyr RTOS code completion. Generated embedded code must be reviewed, compiled, and tested before use.