qwen3-event-extraction-exp5.1

qwen3-event-extraction-exp5.1

Attribute-extraction model for NGEC (Next Generation Event Coder), the pipeline behind the POLECAT political event dataset. Given a document and a PLOVER event type definition, it extracts the actor, recipient, date, location, and an anchor quote for every instance of that event type in the text, as JSON.

This is a 2026 retraining (internally: exp5.1) of ahalt/event-attribute-extractor and supersedes it — position-corrected LLM-judge evaluation against ECAV gold labels shows +4pp actor accuracy and +1pp joint (actor+recipient) accuracy, and a +4pp joint win rate against gold. See "Evaluation" below.

This model was trained on a different prompt format than the original event-attribute-extractor. Prompting it in the old format does not raise an error, but will return somewhatworse spans. If you use this model through the NGEC package, ngec.attribute_model.AttributeModel handles this automatically by keying the prompt format off the model name (see KNOWN_PROMPT_FORMATS). If you use it standalone, use the format below, not the one on the old model card.

Example usage with vLLM

Load the model and tokenizer

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

model = LLM(model="ahalt/qwen3-event-extraction-exp5.1",
            enable_prefix_caching=True,
            max_model_len=8000,
            gpu_memory_utilization=0.80)

tokenizer = AutoTokenizer.from_pretrained("ahalt/qwen3-event-extraction-exp5.1")

sampling_params = SamplingParams(
    temperature=0.5,       # Greedy decoding breaks Qwen
    top_p=0.8,             # Qwen3 non-thinking recommendation
    top_k=20,              # Qwen3 recommendation
    presence_penalty=1.5,  # Recommended for quantized models
    min_p=0.0,
    max_tokens=2048,       # this model runs longer than the old one's 1024
)

Prompt setup

The system prompt states the output format and the extraction rules. The user message puts the whole event-type definition — main definition, sub-event (mode), and any special instructions — inline after ## Event Type:, as a single string; there is no closing "extract the attributes" instruction, because the system prompt already carries it.

system_content = """Given the event type definition below, find all instances of that event in the document and extract their attributes as JSON.

OUTPUT FORMAT:
[
  {
    "event_type": "EVENT_TYPE",
    "anchor_quote": "exact 5-15 word quote from text",
    "actor": "who performed action OR N/A",
    "recipient": "who was targeted OR N/A",
    "date": "when occurred OR N/A",
    "location": "where occurred OR N/A"
  }
]

RULES:
- All values must be exact spans copied from the text. Do not rephrase.
- ACTOR: The person, group, or entity who performed the action. Use N/A only if truly unknown/unstated. Descriptions like "gunman" or "suicide bomber" ARE valid actors.
- LOCATION: Use the most specific named place (city > region > country).
- Use short, concise spans. Omit articles (a/an/the) and unnecessary context.
- Multiple values: separate with semicolons.
- Return [] if no events of the specified type are present.
- Follow any Special Instructions provided with the event type definition."""


def make_prompt(doc, event_type, event_def, tokenizer, mode_def=None, extraction_notes=None):
    definition = f"## Event: **{event_type}**: {event_def}"
    if mode_def:
        definition += f" ## Specific Sub-Event: {mode_def}"
    if extraction_notes:
        definition += f" ## Special Instructions: {extraction_notes}"
    user_content = f"## Document: {doc}\n\n## Event Type: {definition}"

    messages = [
        {"role": "system", "content": system_content},
        {"role": "user", "content": user_content},
    ]
    return tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
        enable_thinking=False,
    )

Event type definitions come from the PLOVER codebook (PLOVER_structured_codebook_updated.csv in the NGEC package) — this model was trained to expect the definition text, not just the bare type name, after ## Event Type:.

Example

text = """KYIV, Ukraine (AP) — Ukraine's anti-corruption agencies said they had uncovered a major graft scheme involving inflated military procurement contracts, just two days after Ukraine's parliament voted to restore the agencies' independence.

In a joint statement published Saturday on social media, the National Anti-Corruption Bureau (NABU) and the Specialized Anti-Corruption Prosecutor's Office (SAPO) said the suspects had taken bribes in a scheme that used state funds to buy drones and other military equipment at inflated prices.

"The essence of the scheme was to conclude state contracts with supplier companies at deliberately inflated prices," the statement said, adding that offenders had received kickbacks of up to 30% of the contracts' value."""

event_type = "INVESTIGATE"
event_def = ("Investigate, charge, or prosecute a person or organization for "
             "wrongdoing, corruption, or crime.")

prompt = make_prompt(text, event_type, event_def, tokenizer)
output = model.generate(prompt, sampling_params=sampling_params)
response = output[0].outputs[0].text.strip()

# [{"event_type": "INVESTIGATE",
#   "anchor_quote": "uncovered a major graft scheme involving inflated military procurement contracts",
#   "actor": "National Anti-Corruption Bureau (NABU); Specialized Anti-Corruption Prosecutor's Office (SAPO)",
#   "recipient": "suspects",
#   "date": "Saturday",
#   "location": "Ukraine"}]

Training

Fine-tuned from Qwen/Qwen3-0.6B for 1 epoch on a synthetic attribute-extraction dataset built from PLOVER-coded news documents, using the prompt format shown above. See the paper for the full data-generation and training methodology.

Caveats

• Recipient extraction is harder than actor extraction across both this model and its predecessor — see the per-event-type breakdown in the paper.
• Bilateral events (e.g. Clash) confuse actor/recipient roles more often than unilateral ones: the model has no training signal for which side is "the" actor when the codebook forces an asymmetric label onto a symmetric event.
• Sample at temperature=0.5. Greedy decoding degrades this model, consistent with Qwen3's own recommendation against greedy decoding for non-thinking mode.