sgym-marketing-1.5b

sgym-marketing-1.5b

A Qwen2.5-1.5B-Instruct model fine-tuned with Direct Preference Optimization (DPO) for marketing skill applications. Part of the SkillGym project for training LLMs to apply domain skills via reinforcement learning.

Model Description

This model is the result of the SkillGym training pipeline, which trains LLMs to apply domain-specific marketing skills. The base Qwen2.5-1.5B-Instruct model was fine-tuned using DPO on preference pairs where the "chosen" response follows a structured marketing skill template and the "rejected" response is a generic unguided output.

The model covers three marketing skill domains: CRO (Conversion Rate Optimization), Copywriting, and Cold Email outreach.

Training Details

• Base model: Qwen/Qwen2.5-1.5B-Instruct
• Method: Direct Preference Optimization (DPO), reference-free
• LoRA config: r=4, alpha=8, dropout=0.2
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• DPO pairs: 17 (from 3 marketing skills: CRO, copywriting, cold-email)
• Epochs: 10
• Beta: 0.05
• Training time: ~4 minutes on NVIDIA RTX 4050

Evaluation Results

Evaluated on 30 marketing tasks using a 6-dimension composite scoring system:

• 17/30 tasks improved
• Per-skill: copywriting -0.021, cold-email +0.021, CRO +0.020
• Best improvements: copy_007 (+0.180), cro_008 (+0.171), cro_009 (+0.126)

Skills Covered

1. CRO (Conversion Rate Optimization): Landing page analysis with heuristic evaluation, A/B test prioritization
2. Copywriting: Website copy creation using PAS/AIDA frameworks, brand voice matching
3. Cold Email: B2B outreach sequences with personalization, objection handling

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("4MLabs/sgym-marketing-1.5b")
tokenizer = AutoTokenizer.from_pretrained("4MLabs/sgym-marketing-1.5b")

messages = [
    {"role": "system", "content": "You are a marketing expert specializing in CRO."},
    {"role": "user", "content": "Analyze this landing page for conversion optimization opportunities: [landing page URL or description]"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation

If you use this model, please cite:

@article{skillgym2026,
  title={SkillGym: Training LLMs to Apply Domain Skills via Reinforcement Learning},
  author={Michiel Horstman},
  year={2026}
}

License

Apache 2.0

Acknowledgments

Built on Qwen2.5-1.5B-Instruct by Alibaba Cloud.