Qwen2.5-0.5B-Instruct

Qwen/Qwen2.5-0.5B-Instruct — MobileTransformers package

On-device (Android) package exported from Qwen/Qwen2.5-0.5B-Instruct with MobileTransformers.

What this package can do

• core — shared files every other group needs
• inference — generate or score on device
• train — fine-tune on device, then merge the adapter back into the base weights
• rag — retrieve over documents you ingest, and ground answers in them

Fine-tuning method

• lora — LoRA — low-rank adapters on the attention projections.
• Rank: 8
• Adapted modules: q_proj, v_proj

Provenance

• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Selected task: text-generation-with-past
• Quantization: int4
• Toolchain: optimum-onnx 0.1.0, transformers 4.57.6, ort-training 1.23.0+cpu

Licenses

• Framework: not declared in this package — see the repository
• Base model weights: see the base model above (this package redistributes an export of those weights, so their terms govern its contents)

Android runtime

• Minimum API: 28
• Required ABIs: any

Variants

Default variant: cpu-int4.

Running this model

This is a MobileTransformers package, not a plain Hugging Face model: it is a manifest plus per-variant ONNX stages and a weight-handoff map. transformers, optimum and plain onnxruntime cannot load it. Use the framework:

https://github.com/martinkorelic/mobiletransformers

// Android — pulls, verifies and installs on first use.
val model = MobileTransformers.fromPretrained(
    context = context,
    repoId  = "mobiletransformers/Qwen2.5-0.5B-Instruct",
)

# Host — download and inspect the package without a device.
mobiletransformers pull --repo-id mobiletransformers/Qwen2.5-0.5B-Instruct

Citation

If you are using this framework for your own work, please cite:

@misc{mobiletransformers2025,
  author       = {Koreli\v{c}, Martin and Pejovi{\'c}, Veljko},
  title        = {MobileTransformers: An On-Device LLM PEFT Framework for Fine-Tuning and Inference},
  year         = {2025},
  howpublished = {\url{https://gitlab.fri.uni-lj.si/lrk/mobiletransformers}}
}