posthog-specialist-qwen2.5-3b

PostHog Specialist Model - Qwen2.5-3B

Model Description

This model is a specialized version of Qwen/Qwen2.5-3B-Instruct, fine-tuned on PostHog-specific Q&A data to assist with PostHog-related questions, support, debugging, documentation workflows, CLI usage, and product behavior.

Base Model: Qwen/Qwen2.5-3B-Instruct
Training Method: LoRA (Low-Rank Adaptation)
Dataset: AngelRaychev/posthog-specialist-dataset (303 samples from GitHub issues)
License: Apache 2.0

Training Details

Training Configuration

• Method: LoRA fine-tuning
• LoRA Rank: 8
• LoRA Alpha: 16
• Target Modules: q_proj, v_proj
• Epochs: 3
• Batch Size: 1 per device
• Gradient Accumulation: 8 steps (effective batch size: 8)
• Learning Rate: 1e-4
• Max Sequence Length: 1024 tokens
• Optimization: AdamW with BF16 precision
• Gradient Checkpointing: Enabled
• Hardware: a10g-large (24GB GPU)
• Training Time: ~6 minutes (350 seconds)

Training Results

• Initial Loss: 3.246
• Final Loss: 2.21
• Initial Accuracy: 0.438
• Final Accuracy: 0.528
• Eval Loss (Epoch 3): 2.21

Dataset

The model was trained on a custom dataset of 303 PostHog Q&A pairs extracted from:

• GitHub issues with high engagement
• Product feature discussions
• User questions and maintainer responses

Dataset Split:

• Training: 272 samples
• Test: 31 samples

Dataset Format: Conversational (messages with user/assistant roles)

Use Cases

This model is designed to help with:

• PostHog-specific support questions
• Debugging PostHog integrations
• Understanding PostHog features (feature flags, session replay, analytics)
• CLI usage and setup
• SDK integration and product behavior
• Documentation workflows

How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import AutoPeftModelForCausalLM
import torch

# Load model and tokenizer
model = AutoPeftModelForCausalLM.from_pretrained(
    "AngelRaychev/posthog-specialist-qwen2.5-3b",
    device_map="auto",
    torch_dtype=torch.bfloat16,
)
tokenizer = AutoTokenizer.from_pretrained("AngelRaychev/posthog-specialist-qwen2.5-3b")

# Prepare question
messages = [
    {"role": "user", "content": "How do I install PostHog using npm?"}
]

# Generate response
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Limitations

• The model was trained on a relatively small dataset (303 samples)
• Focus is primarily on general PostHog questions from GitHub issues
• May not be up-to-date with the very latest PostHog features
• Works best for English language queries
• Limited to 1024 token context window

Future Improvements

• Expand dataset with more diverse PostHog documentation
• Include PostHog SDK code examples
• Add CLI usage patterns
• Incorporate PostHog handbook content
• Increase training data to 1000+ samples
• Fine-tune on specific PostHog workflows (wizards, sourcemaps, etc.)

Model Card

• Developed by: AngelRaychev
• Model type: Causal Language Model (LoRA adapter)
• Base model: Qwen/Qwen2.5-3B-Instruct
• Language(s): English
• Finetuned from: Qwen/Qwen2.5-3B-Instruct
• License: Apache 2.0

Training Infrastructure

Trained on Hugging Face Jobs infrastructure using:

• Hardware: a10g-large (24GB GPU)
• Framework: TRL (Transformers Reinforcement Learning)
• Libraries: transformers, peft, torch, datasets, accelerate
• Experiment tracking: Trackio

Citation

@misc{posthog-specialist-qwen2.5-3b,
  author = {AngelRaychev},
  title = {PostHog Specialist Model - Qwen2.5-3B},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/AngelRaychev/posthog-specialist-qwen2.5-3b}}
}

Acknowledgments

• Base model: Qwen/Qwen2.5-3B-Instruct
• Training framework: TRL
• Data source: PostHog GitHub Repository