Qwen2.5-Coder-7B-ABAP

Qwen2.5-Coder-7B-ABAP

A fine-tuned version of Qwen/Qwen2.5-Coder-7B-Instruct specialized for SAP ABAP development.

Training Scripts

Two scripts are provided depending on your hardware:

Quick Start (8GB VRAM)

pip install torch transformers trl peft datasets accelerate bitsandbytes
huggingface-cli login
python train_abap_qlora.py

Quick Start (24GB+ VRAM)

pip install torch transformers trl peft datasets accelerate bitsandbytes
huggingface-cli login
python train_abap.py

Datasets

Training Configurations

Full LoRA (24GB+ VRAM) — train_abap.py

QLoRA (8GB VRAM) — train_abap_qlora.py

Usage

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained("SpaceArm/Qwen2.5-Coder-7B-ABAP")
tokenizer = AutoTokenizer.from_pretrained("SpaceArm/Qwen2.5-Coder-7B-ABAP")

messages = [
    {"role": "system", "content": "You are an expert SAP ABAP developer."},
    {"role": "user", "content": "Write an ABAP class that reads data from table MARA and displays it in an ALV grid."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Capabilities

• ✅ ABAP report writing (REPORT, WRITE, LOOP, SELECT)
• ✅ Object-oriented ABAP (classes, interfaces, inheritance)
• ✅ Internal tables and data manipulation
• ✅ ALV grid programming
• ✅ Function modules and BAPIs
• ✅ ABAP security vulnerability detection
• ✅ Modern ABAP syntax and best practices
• ✅ CDS views and RAP concepts

Evaluation

Evaluate against timkoehne/LLM-ABAP-Code-Generation-Benchmark (HumanEval adapted for ABAP).

OOM Troubleshooting

If you hit out-of-memory on 8GB VRAM:

1. Reduce max_length from 1024 → 512 in train_abap_qlora.py
2. Ensure no other GPU processes are running (nvidia-smi)
3. Close browser tabs / desktop apps using GPU
4. Set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True before running

Research Background

ABAP is a low-resource programming language — while included in large code corpora like The Stack v2, training data is scarce compared to Python/Java. This model uses approaches from:

• Low-resource PL fine-tuning (arxiv:2501.19085): Fine-tuning on domain-specific instruction data improves performance on underrepresented languages
• Qwen2.5-Coder (arxiv:2409.12186): Best available base model with ABAP exposure in its 92-language pretraining corpus
• QLoRA (arxiv:2305.14314): 4-bit quantized training enabling 7B model fine-tuning on consumer GPUs