nci-technique-classifier-v2

NCI Technique Classifier v2

Multi-label propaganda technique classifier for the NCI (News Content Intelligence) Protocol.

Model Description

This model classifies text into 18 propaganda techniques as part of a two-stage pipeline:

• Stage 1: Binary detection (synapti/nci-binary-detector-v2) determines if propaganda exists
• Stage 2: This model identifies which specific techniques are used

Techniques Detected

Training

• Base Model: answerdotai/ModernBERT-base
• Dataset: synapti/nci-propaganda-production (19,581 train, 1,727 val, 1,729 test)
• Loss: Focal Loss (gamma=2.0) with class weights for imbalanced techniques
• Epochs: 5
• Batch Size: 16
• Learning Rate: 2e-5
• Hardware: NVIDIA A10G GPU

Performance

Per-Technique Performance (selected)

Usage

With Transformers

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model = AutoModelForSequenceClassification.from_pretrained("synapti/nci-technique-classifier-v2")
tokenizer = AutoTokenizer.from_pretrained("synapti/nci-technique-classifier-v2")

text = "The radical left is DESTROYING our great nation!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.sigmoid(outputs.logits)[0]

# Get techniques above threshold
threshold = 0.5
techniques = list(model.config.id2label.values())
detected = [(techniques[i], probs[i].item()) for i in range(len(techniques)) if probs[i] > threshold]
print(detected)

With NCI Protocol

from nci.transformers.two_stage_pipeline import TwoStagePipeline

pipeline = TwoStagePipeline.from_pretrained(
    binary_model="synapti/nci-binary-detector-v2",
    technique_model="synapti/nci-technique-classifier-v2",
)

result = pipeline.analyze("The radical left is DESTROYING our great nation!")
print(f"Has propaganda: {result.has_propaganda}")
print(f"Techniques: {[t.name for t in result.techniques if t.above_threshold]}")

ONNX Inference

ONNX model available in onnx/model.onnx for faster inference (~1.25x speedup).

import onnxruntime as ort
import numpy as np
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("synapti/nci-technique-classifier-v2")
session = ort.InferenceSession("onnx/model.onnx")

text = "WAKE UP AMERICA!"
inputs = tokenizer(text, return_tensors="np", truncation=True, max_length=512)

outputs = session.run(None, {
    "input_ids": inputs["input_ids"],
    "attention_mask": inputs["attention_mask"]
})
probs = 1 / (1 + np.exp(-outputs[0]))  # sigmoid

Limitations

• Trained primarily on English news articles
• May not generalize well to social media or other domains
• Threshold of 0.5 may need adjustment for specific use cases
• Multi-label classification means multiple techniques can be detected per text

Citation

@misc{nci-technique-classifier-v2,
  author = {Synapti},
  title = {NCI Technique Classifier v2},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/synapti/nci-technique-classifier-v2}
}

License

MIT License