Downloads · 30 days
5
6% of all-time downloads
kiranpg/qwen2.5-ocamler-sft-model-2025-01-09-1733
qwen2.5-ocamler-sft-model-2025-01-09-1733 is a text generation model from kiranpg. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a fine-tuned version of Qwen/Qwen2.5-Coder-1.5B-Instruct specialized for generating OCaml code.
Downloads · 30 days
5
6% of all-time downloads
All-time downloads
79
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.1 GB · 99%
From the Hugging Face model README
This model is a fine-tuned version of Qwen/Qwen2.5-Coder-1.5B-Instruct specialized for generating OCaml code.
| Parameter | Value |
|---|---|
| Batch Size | 16 |
| Gradient Accumulation Steps | 8 |
| Effective Batch Size | 128 |
| Learning Rate | 2e-5 |
| Number of Epochs | 3 |
| Max Sequence Length | 1024 |
| LR Scheduler Type | cosine |
| Warmup Ratio | 0.01 |
| Weight Decay | 0.05 |
| Max Grad Norm | 1.0 |
| Optimizer | adamw_8bit |
| Dataloader Num Workers | 4 |
| Parameter | Value |
|---|---|
| LoRA Rank (r) | 64 |
| LoRA Alpha | 128 |
| LoRA Dropout | 0.1 |
| Parameter | Value |
|---|---|
| Logging Steps | 1 |
| Eval Steps | 5 |
| Save Steps | 10 |
| Save Total Limit | 5 |
| Metric | Value |
|---|---|
| Final Train Loss | 0.2599 |
| Final Eval Loss | 0.2768 |
| Total Steps | 66 |
| Training Time | 16m 38s |

The training curves show:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "kiranpg/qwen2.5-ocamler-sft-model-2025-01-09-1733"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [
{"role": "user", "content": "Write an OCaml function to compute the factorial of a number."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
This model is designed for generating OCaml code solutions given natural language problem descriptions. It has been fine-tuned on OCaml programming problems to improve its ability to produce correct, idiomatic OCaml code.
Trained using TRL's SFTTrainer with completion-only training (prompts masked from loss).