qwen3-8b-karma-v3

Qwen3-8B Karma v3 (Authentic DPO)

A fine-tuned Qwen3-8B model trained to generate Reddit comments that match specific karma score targets. This model combines supervised fine-tuning with Direct Preference Optimization (DPO) to learn what makes Reddit comments successful.

Model Description

This is the v3 release featuring:

• Expanded subreddit coverage: Trained on 33 diverse subreddits
• Persona steering: Includes commenter personality descriptions for style control
• Authentic DPO: Uses stricter preference pairs filtered for authentic, high-signal responses
• Adaptive thresholds: Per-subreddit score gap requirements based on community karma distributions

Training Pipeline

Stage 1 - Supervised Fine-Tuning (SFT)

• Task: Learn to predict karma scores and generate contextually appropriate replies
• Input: Thread context (subreddit, post title, body, parent comments, target score)
• Output: Reply text matching the target karma level

Stage 2 - Direct Preference Optimization (DPO)

• Creates preference pairs from sibling comments (same parent, different scores)
• Higher-scored comments = "chosen", lower-scored = "rejected"
• Filters for meaningful score gaps (adaptive per subreddit)
• Trained with persona conditioning for style control

Subreddits

Trained on ~3.4M comments from 33 subreddits including:

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("verque-app/qwen3-8b-karma-v3")
tokenizer = AutoTokenizer.from_pretrained("verque-app/qwen3-8b-karma-v3")

# Format prompt for karma-targeted generation
prompt = """<|im_start|>user
Subreddit: r/AskReddit
Title: What's something that sounds fake but is actually true?
Post Score: 15420
Persona: Enthusiastic storyteller who uses vivid details
Task: write a Reddit reply targeting this karma score
Target Score: high<|im_end|>
<|im_start|>assistant
<think>

</think>

"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))

Score Targets

The model uses log-normalized score buckets:

• low: 1-10 karma
• medium: 10-100 karma
• high: 100-1000 karma
• top: 1000+ karma

Persona Steering (v3 Feature)

The Persona: field allows you to control the writing style and tone of generated comments. This is a key feature of v3 that enables fine-grained control over the model's output personality.

Example personas:

• "Blunt and pragmatic, gives actionable advice without sugarcoating."
• "Skeptical and demands evidence, uses dry humor."
• "Enthusiastic expert who gets excited about technical details."
• "Rude, obnoxious, toxic" - for testing edge cases
• "Helpful and empathetic, validates feelings before giving advice."

The persona field steers the conversation tone while the target score influences content quality/engagement level.

Persona Examples

Character Archetypes

Use these to simulate different Reddit personality types responding to a post:

Restatement Styles

Use these to generate variations of the same message with different tones:

Example Output

Post: "Struggling with consistent social media content" (r/AskMarketing)

Doomer Dan:

It's too late. You should have hired a designer or a copywriter when you were just starting out.

The Normie:

I'm not a fragrance brand, but for my own business I use a combination of free stock photos/videos and free AI tools. Been doing this for a couple of months and it works. If it was me I'd say go for it.

Anecdote Andy:

Well in MY experience, the hardest part is consistency. It is not easy to just post random stuff because you feel like it, you need to post stuff that is connected to your brand. I have a few accounts (Instagram and TikTok) and each one is slightly different, but they are all connected to my brand.

Hot Take Henry:

The problem is that AI tools will not be able to do that. They will give you random images that are not related to your brand.

API Usage (vLLM)

If serving with vLLM, you can use these curl examples:

Generate a Reddit Comment with Persona

curl -X POST http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "<|im_start|>user\nSubreddit: r/AskReddit\nTitle: What'\''s something the older generations were actually right about?\nPersona: Thoughtful and nostalgic, shares personal anecdotes\nPost Score: 423\nTask: write a Reddit reply targeting this karma score\nTarget Score: 6.5<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n",
    "temperature": 0.7,
    "max_tokens": 1024,
    "stop": ["<|im_end|>"]
  }'

Generate a Reply to an Existing Comment

curl -X POST http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "<|im_start|>user\nSubreddit: r/technology\nTitle: Apple announces new M4 chip\nBody: The new M4 chip promises 50% better performance than M3.\nPersona: Helpful explainer who provides context\nPost Score: 500\nParent 1: 50% sounds impressive but what'\''s the real-world difference?\nParent 1 Score: 200\nTask: write a Reddit reply targeting this karma score\nTarget Score: 4.6<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n",
    "temperature": 0.7,
    "max_tokens": 1024,
    "stop": ["<|im_end|>"]
  }'

Predict Karma Score for a Comment

curl -X POST http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "<|im_start|>user\nSubreddit: r/technology\nTitle: Apple announces new M4 chip\nBody: The new M4 chip promises 50% better performance than M3.\nPost Score: 500\nTask: predict the karma score for this reply\nReply: Sounds cool but I'\''ll wait for independent benchmarks before getting excited.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n",
    "temperature": 0.3,
    "max_tokens": 64,
    "stop": ["<|im_end|>"]
  }'

Training Details

Hyperparameters

Stage 1 (SFT)

• Learning rate: 2e-4
• Batch size: 16 (effective)
• Max steps: 10,000
• LoRA rank: 16, alpha: 32
• Max sequence length: 1024

Stage 2 (DPO)

• Learning rate: 2e-6
• DPO beta: 0.3
• Batch size: 8 (effective)
• Max steps: 450
• Warmup ratio: 0.1

Hardware

• 4x NVIDIA A10G GPUs (g5.12xlarge)
• Training precision: bf16
• Flash Attention 2 enabled

Limitations

• Trained primarily on English Reddit content
• May reflect biases present in Reddit communities
• Score prediction is probabilistic, not deterministic
• Best results on subreddits similar to training data

Intended Use

• Research on social media dynamics and content virality
• Understanding what makes online content engaging
• Creative writing assistance for authentic-sounding responses
• NOT intended for spam, manipulation, or deceptive practices

Citation

@misc{qwen3-karma-v3,
  title={Qwen3-8B Karma v3: Reddit Comment Generation with DPO},
  author={Verque},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/verque-app/qwen3-8b-karma-v3}
}

Find out more at https://verque.app