Qwen3-4B-populism

Qwen3-4B Fine-tuned for Populism Detection

Model Description

This model is a fine-tuned version of Qwen/Qwen3-4B trained to identify fine-grained forms of populism in political discourse. The model performs 4-way classification to detect:

• (a) No populism - Sentences without populist rhetoric
• (b) Anti-elitism - Negative invocations of "elites"
• (c) People-centrism - Positive invocations of the "people"
• (d) Both - Sentences combining anti-elitism and people-centrism

The model was fine-tuned using LoRA (Low-Rank Adaptation) for parameter-efficient training on political speeches from Donald Trump's 2016 presidential campaign.

Model Details

Model Type

• Base Model: Qwen/Qwen3-4B (4B parameters)
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Task: Multi-class Text Classification (4 categories)
• Language: English
• Domain: Political discourse

Training Details

LoRA Configuration:

• Rank (r): 16
• Alpha: 32
• Dropout: 0.05
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Bias: None
• Task Type: CAUSAL_LM

Training Hyperparameters:

• Epochs: 3
• Batch Size: 8 (per device)
• Gradient Accumulation Steps: 4
• Learning Rate: 2e-4
• Learning Rate Scheduler: Cosine with 0.1 warmup ratio
• Optimizer: paged_adamw_32bit
• Weight Decay: 0.001
• Max Gradient Norm: 1.0
• Max Sequence Length: 300 tokens
• Quantization: 4-bit (NF4) with double quantization
• Mixed Precision: FP16/BF16
• Total Training Steps: 1,476

Training Loss Progression:

• Step 1: 4.9333
• Step 100: 0.4583
• Step 500: 0.0098
• Step 1000: 0.003
• Step 1476: ~0.001

Training Data

Dataset: coastalcph/populism-trump-2016

Data Source: Sentence-level annotations from Donald Trump's 2016 presidential campaign speeches.

Label Distribution (Original):

• (a) No populism: 92%
• (b) Anti-elitism: 4%
• (c) People-centrism: 2%
• (d) Both: 2%

Data Preprocessing:

• Sentences truncated to maximum 48 words
• 5x upsampling applied to minority classes (b, c, d) to address class imbalance
• Tokenized with max_length=300
• Shuffled with seed=42

Populism Definition: Following political science literature, populism is defined as an anti-elite discourse in the name of "the people", characterized by:

1. Anti-elitism: Negative references to elite groups
2. People-centrism: Positive references to ordinary citizens or "the people"

How to Use

Option 1: Load Merged Model (Recommended)

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

# Load model and tokenizer
model_name = "armaniii/Qwen3-4B-populism"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    torch_dtype="auto"
)

# Create pipeline
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer
)

Inference Example

# System prompt
SYSTEM_PROMPT = """You are a helpful AI assistant with expertise in identifying populism in public discourse.

Populism can be defined as an anti-elite discourse in the name of the "people". In other words, populism emphasizes the idea of the common "people" and often positions this group in opposition to a perceived elite group.

There are two core elements in identifying populism: (i) anti-elitism, i.e., negative invocations of "elites", and (ii) people-centrism, i.e., positive invocations of the "people".
"""

# Instruction template
INSTRUCTION = """You must classify each sentence in one of the following categories:

(a) No populism.
(b) Anti-elitism, i.e., negative invocations of "elites".
(c) People-centrism, i.e., positive invocations of the "People".
(d) Both people-centrism and anti-elitism populism.
"""

# Example sentence
sentence = "We need to stand up against the corrupt establishment that has betrayed the American people."

# Build prompt
conversation = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": INSTRUCTION + f'\n\nWhich is the most relevant category for the sentence: "{sentence}"?'}
]

# Apply chat template
prompt = tokenizer.apply_chat_template(
    conversation=conversation,
    tokenize=False,
    add_generation_prompt=True
)

# Generate response
response = pipe(
    prompt,
    do_sample=False,
    max_new_tokens=256,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.eos_token_id,
)

print(response[0]['generated_text'])
# Expected output: "I would categorize this sentence as (d)..."

Expected Output Format

The model responds in the following format:

I would categorize this sentence as (X)

[Explanation of the classification decision]

Where X is one of: a, b, c, or d.

Performance

Detailed performance metrics and comparisons are available in the original paper (see Citation section below).

Key Findings:

• Fine-tuned models significantly outperform zero-shot instruction-tuned LLMs
• The model shows strong performance on in-domain political discourse
• Cross-context evaluation demonstrates reasonable generalization to European political speeches
• LoRA fine-tuning provides efficient adaptation with minimal trainable parameters

Limitations and Bias

Training Data Limitations:

• Trained primarily on Donald Trump's 2016 campaign speeches
• May not generalize equally well to other political contexts, time periods, or speakers
• Performance may vary on non-US political discourse

Class Imbalance:

• Original data is highly imbalanced (92% "No populism")
• Upsampling (5x) applied to minority classes during training
• Model may still show bias toward the majority class

Domain Specificity:

• Optimized for sentence-level classification
• Performance on longer texts may require sentence segmentation
• Best suited for political discourse and campaign rhetoric

Ethical Considerations:

• This model provides automated analysis and should not be the sole basis for political judgments
• Results should be interpreted by domain experts
• May reflect biases present in the training data

Intended Use

Primary Use Cases:

• Research in political science and computational social science
• Analysis of populist rhetoric in political campaigns
• Educational purposes in understanding populist discourse
• Automated annotation of political speech datasets

Out-of-Scope Use:

• Making definitive political judgments about individuals or parties
• Real-time moderation or censorship of political speech
• Use without human oversight in sensitive political contexts

Citation

If you use this model, please cite the original paper:

@misc{chalkidis2025populism,
    title={Identifying Fine-grained Forms of Populism in Political Discourse: A Case Study on Donald Trump's Presidential Campaigns},
    author={Chalkidis, Ilias and Brandl, Stephanie and Aslanidis, Paris},
    year={2025},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2507.19303},
    doi={10.48550/arXiv.2507.19303}
}

Paper: Identifying Fine-grained Forms of Populism in Political Discourse

Dataset: coastalcph/populism-trump-2016

Acknowledgments

This model was developed based on the research by:

• Ilias Chalkidis (University of Copenhagen)
• Stephanie Brandl (University of Copenhagen)
• Paris Aslanidis (Aristotle University of Thessaloniki)

The fine-tuning was performed using:

• Base Model: Qwen/Qwen3-4B by Alibaba Cloud
• Dataset: coastalcph/populism-trump-2016
• PEFT Library: Hugging Face PEFT for LoRA implementation
• Training Framework: Hugging Face Transformers and TRL

Model Card Authors

[Your Name/Organization]

Model Card Contact

[Your Contact Information]

License

This model is released under the Apache 2.0 License, consistent with the Qwen3-4B base model.

Responsible Use: This model is intended for research and educational purposes. Users should be aware of the limitations and biases described above and use the model responsibly with appropriate human oversight.

---

Related Resources:

• 📄 Paper
• 🤗 Dataset
• 💻 GitHub Repository
• 🎯 Demo Space