Qwen2.5-OCamler-1.5B-Instruct

Qwen2.5-OCamler-1.5B-Instruct

This model is a fine-tuned version of Qwen/Qwen2.5-Coder-1.5B-Instruct specialized for generating OCaml code.

Model Details

• Base Model: Qwen/Qwen2.5-Coder-1.5B-Instruct
• Fine-tuning Method: Generative Representational Preference Optimization (GRPO) with LoRA
• Training Dataset: kiranpg/ocaml-training-problems
• Training Code: ocamler-grpo

Training Configuration

GRPO Parameters

LoRA Configuration

Training Settings

GRPO-Specific Parameters

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "kiranpg/Qwen2.5-OCamler-1.5B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

messages = [
    {"role": "user", "content": "Write an OCaml function to compute the factorial of a number."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Use

This model is designed for generating OCaml code solutions given natural language problem descriptions. It has been fine-tuned on OCaml programming problems using GRPO with real-time feedback from the OCaml compiler and test suite to improve its ability to produce correct, idiomatic OCaml code.

Limitations

• The model may not always produce syntactically correct OCaml code
• Complex algorithmic problems may require multiple attempts
• The model works best with clear, well-specified problem descriptions

Training Infrastructure

Trained using TRL's GRPOTrainer with OCaml compiler verification for rewards. The reward system uses a graduated structure:

• Type checking: 25% (partial credit scaled by error count)
• Compilation: 10% (partial credit based on type check)
• Tests: 65% (all-or-nothing for passing)