poc-simorg-coder-30b-a3b-qwen3-lora-v0.1

Simorg Qwen3-Coder-30B-A3B-Instruct (fine-tuned)

Hugging Face: simorg-platform/poc-simorg-coder-30b-a3b-qwen3-lora-v0.1

Fine-tuned Qwen/Qwen3-Coder-30B-A3B-Instruct for the Simorg programming language.

This repository contains the merged full weights, LoRA adapter, GGUF quantizations, training data, and Ollama Modelfiles.

Model summary

Repository layout

safetensors/          Merged full model (Hugging Face / transformers)
gguf/                 GGUF quantizations (llama.cpp, Ollama, LM Studio)
lora/                 LoRA adapter only (PEFT)
training-data/        JSON instruction dataset used for SFT
ollama/               Modelfiles for Ollama import
LICENSE               Apache License 2.0
NOTICE                Attribution and modification notice

See MANIFEST.md for the complete file list.

Usage

Transformers (Safetensors)

Point at the safetensors/ folder or download this repo and load from safetensors/:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "simorg-platform/poc-simorg-coder-30b-a3b-qwen3-lora-v0.1"  # safetensors/ subfolder or repo root
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

If the Hub repo keeps weights under safetensors/, pass the subpath:

model_path = "simorg-platform/poc-simorg-coder-30b-a3b-qwen3-lora-v0.1/safetensors"

GGUF (llama.cpp)

Recommended quant: gguf/qwen3-coder-simorg-Q4_K_M.gguf (~17 GB).

llama-cli -m gguf/qwen3-coder-simorg-Q4_K_M.gguf -p "Your prompt"

Ollama

After cloning this repository:

cd ollama
ollama create simorg-qwen3-q4 -f Modelfile-qwen-3-Q4-K-M
ollama run simorg-qwen3-q4

Modelfiles use FROM ../gguf/... — run ollama create from the ollama/ directory.

LoRA adapter (PEFT)

Load the adapter from lora/ on top of the base model:

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

base = "Qwen/Qwen3-Coder-30B-A3B-Instruct"
model = AutoPeftModelForCausalLM.from_pretrained("path/to/lora", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("path/to/lora", trust_remote_code=True)

Training data

Instruction JSON in training-data/ with question and answer fields. Used for supervised fine-tuning on Simorg syntax and patterns.

Training details

• Method: QLoRA SFT with TRL SFTTrainer, assistant-only loss
• Base: Qwen/Qwen3-Coder-30B-A3B-Instruct
• LoRA rank: 16 (attention), 8 (expert MLP); use_rslora=True
• Learning rate: 1e-4, cosine schedule, 3 epochs
• Max length: 4096 tokens during training

License and attribution

This model is a derivative work of Qwen3-Coder-30B-A3B-Instruct by Alibaba Cloud, licensed under the Apache License 2.0.

• See LICENSE for the full license text.
• See NOTICE for modification and attribution details.

You may use, modify, and redistribute this model under the terms of Apache 2.0, including commercial use, provided you include the license and state significant modifications.

This repository does not grant permission to use Qwen or Alibaba trademarks in a way that implies endorsement.

Citation

@misc{qwen3technicalreport,
  title={Qwen3 Technical Report},
  author={Qwen Team},
  year={2025},
  eprint={2505.09388},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2505.09388},
}

Disclaimer

Model outputs may be incorrect or incomplete. This is a PoC model, fine-tuned on top of a PoC version of Simorg Programming Language! Please don't use it in a production environment and wait for the first LTS version.