ConflLlama-NER

ConflLlama-NER: Domain-Specific Named Entity Recognition for Conflict Events

<p align="center"> <img src="images/logo.png" alt="ConflLlama-NER Logo" width="300"/> </p>

---

⚠️ Important: Read Before Using

This model requires exact prompt formatting to work correctly. Please read the Critical: Inference & Prompt Formatting section below before attempting to use this model. Using incorrect prompts will result in poor performance or hallucinations.

---

ConflLlama-NER is a specialized large language model fine-tuned for Named Entity Recognition (NER) in conflict event narratives. Built upon Llama-3.1 8B Instruct, this model identifies and classifies three critical entity types in political violence texts:

• Source (B-S/I-S): The actor or group initiating the action
• Target (B-T/I-T): The recipient or victim of the action
• Related (B-R/I-R): Other significant entities mentioned in the context

This model is part of the ConflLlama research project and complements the ConflLlama attack classification model.

---

Key Features

• Specialized Domain Adaptation: Fine-tuned on CAMEO-coded conflict event data for superior performance on political violence narratives
• Multi-Entity Recognition: Simultaneously identifies sources, targets, and related entities with precise boundary detection
• BIO Tagging Scheme: Uses Begin-Inside-Outside annotation for accurate multi-word entity spans
• JSON Output Format: Returns structured entity lists with text, type, and position information
• Efficient Deployment: Available in multiple quantization formats (Q4_K_M, Q8_0, BF16) for various hardware configurations
• Instruction-Tuned: Built on Llama-3.1 Instruct for robust prompt following and reliable output formatting

---

Training Data

Dataset: CAMEO NER Corpus

The model was trained on a custom-annotated dataset derived from conflict event narratives:

• Training Set: 1,094 sentences with entity-level annotations
• Total Tokens: 28,815 tokens
• Annotation Scheme: BIO (Begin-Inside-Outside) tagging
• Entity Distribution:
  • Source entities (B-S/I-S): 3,506 tokens (12.2%)
  • Target entities (B-T/I-T): 3,533 tokens (12.3%)
  • Related entities (B-R/I-R): 536 tokens (1.9%)
  • Non-entity tokens (O): 21,240 tokens (73.7%)

Entity Type Definitions

---

Model Architecture

• Base Model: unsloth/llama-3-8b-Instruct-bnb-4bit
• Fine-tuning Method: QLoRA (Quantized Low-Rank Adaptation)
• Quantization: 4-bit with bitsandbytes
• Maximum Sequence Length: 2048 tokens (reduced from 4096 for NER optimization)
• LoRA Configuration:
  • Rank (r): 8
  • Alpha (lora_alpha): 16
  • Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
  • Dropout: 0
  • Gradient Checkpointing: Enabled (Unsloth optimization)

Training Configuration

• Optimizer: AdamW (torch implementation)
• Learning Rate: 2e-4 with 5% warmup
• Batch Size: 1 per device with 8 gradient accumulation steps (effective batch size: 8)
• Training Steps: 2,000 steps
• Precision: BFloat16 (when supported)
• Hardware: NVIDIA A100-SXM4-40GB GPU
• Memory Footprint: ~6 GB VRAM (optimized for consumer GPUs)

---

Critical: Inference & Prompt Formatting

IMPORTANT: This model requires exact prompt formatting to function correctly. The model was instruction-tuned with a specific template. Deviating from this format will result in poor performance or hallucinations.

Option 1: Using LM Studio / Llama.cpp (Recommended)

System Prompt:

Extract all named entities from the following text. Return them as a JSON list with 'text', 'type', 'start', and 'end' positions.

User Message:

Text: [YOUR ARTICLE TEXT HERE]
Entities: 

Option 2: Direct API/Python Integration

If your platform requires a single concatenated prompt:

prompt = """<|start_header_id|>system<|end_header_id|>

Extract all named entities from the following text. Return them as a JSON list with 'text', 'type', 'start', and 'end' positions.<|eot_id|><|start_header_id|>user<|end_header_id|>

Text: {your_text_here}
Entities: <|eot_id|><|start_header_id|>assistant<|end_header_id|>"""

Example Usage

Input:

Text: Security forces conducted an operation in the northern region. Armed groups fired upon government troops during the incident.
Entities:

Expected Output:

[
  {"text": "Security forces", "type": "B-S", "start": 0, "end": 1},
  {"text": "northern region", "type": "B-T", "start": 6, "end": 7},
  {"text": "Armed groups", "type": "B-S", "start": 9, "end": 10},
  {"text": "government troops", "type": "B-T", "start": 13, "end": 14}
]

---

Installation & Usage

Using Llama.cpp (Recommended for Local Deployment)

# Download the Q4_K_M GGUF model
wget https://huggingface.co/shreyasmeher/confllama-ner-sft-GGUF/resolve/main/model-unsloth-Q4_K_M.gguf

# Run with llama.cpp
./llama-cli -m model-unsloth-Q4_K_M.gguf \
  --system "Extract all named entities from the following text. Return them as a JSON list with 'text', 'type', 'start', and 'end' positions." \
  --prompt "Text: [YOUR TEXT]\nEntities: " \
  --temp 0.3 \
  --n-predict 512

Using Transformers (Python)

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "shreyasmeher/confllama-ner-sft"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Prepare input with correct formatting
text = "Israeli forces raided Palestinian villages in the West Bank."
prompt = f"""<|start_header_id|>system<|end_header_id|>

Extract all named entities from the following text. Return them as a JSON list with 'text', 'type', 'start', and 'end' positions.<|eot_id|><|start_header_id|>user<|end_header_id|>

Text: {text}
Entities: <|eot_id|><|start_header_id|>assistant<|end_header_id|>"""

# Generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.3,
    do_sample=False,
    pad_token_id=tokenizer.eos_token_id
)

result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

Using Unsloth (For Further Fine-tuning)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="shreyasmeher/confllama-ner-sft",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

# Enable inference mode
FastLanguageModel.for_inference(model)

# Generate
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.3)
result = tokenizer.batch_decode(outputs)

---

Intended Use

This model is designed for academic research and analysis in the following domains:

1. Automated Event Coding: Extract structured actor information from conflict narratives
2. Political Violence Research: Identify sources and targets in terrorism and insurgency studies
3. Computational Social Science: Enable large-scale quantitative analysis of conflict data
4. Information Extraction: Build knowledge graphs from unstructured conflict reports
5. Preprocessing for Event Classification: Generate structured inputs for downstream tasks

Example Applications

• Extracting actors from news articles for conflict databases (ICEWS, GDELT, ACLED)
• Building relational networks between conflict actors
• Identifying victim and perpetrator patterns in political violence
• Preprocessing text for the ConflLlama attack classification model

---

Limitations

1. Domain Specificity: Optimized for conflict and political violence texts. Performance may degrade on general-domain NER tasks.

2. Training Data Constraints:

   • Trained on 1,094 sentences (relatively small for modern NER)
   • Class imbalance: "Related" entities (B-R/I-R) are underrepresented (1.9% of tokens)
   • May underperform on rare entity types or novel contexts

3. Entity Boundary Detection:

   • May occasionally split or merge multi-word entities incorrectly
   • Nested entities are not explicitly modeled

4. Temporal Scope: Training data reflects conflict patterns up to the model's knowledge cutoff (January 2025). Emerging actors or tactics may not be recognized.

5. Language: English-only. No support for multilingual entity recognition.

6. Output Parsing: Model outputs JSON, but may occasionally produce malformed JSON under edge cases. Implement robust parsing with error handling.

7. Prompt Sensitivity: Critical limitation - requires exact prompt formatting. Deviations will significantly degrade performance.

---

Ethical Considerations

Responsible Use

1. Research-Only Tool: This model is designed for academic analysis, not operational security decisions or real-time threat assessment.

2. Sensitive Data: The model processes information about real-world violence. Users must:

   • Handle extracted data with appropriate security measures
   • Respect privacy of individuals mentioned in source texts
   • Avoid using entity information to target or profile individuals/groups

3. Bias and Representation:

   • Training data may reflect biases in how conflict events are reported
   • Entity recognition may vary across different regions, actor types, or conflict contexts
   • Do not assume equal performance across all geopolitical contexts

4. Dual-Use Concerns: While designed for research, entity extraction could theoretically support harmful applications:

   • Do not use for surveillance, profiling, or targeting of individuals
   • Do not use to support operational military or security activities
   • Do not use to generate disinformation or manipulate public discourse

5. Transparency: Users should:

   • Clearly disclose when research findings are based on automated NER
   • Report model limitations in publications
   • Validate critical findings with manual review

Accountability

This model is released for research purposes under the Apache 2.0 license. Users are responsible for ensuring their applications comply with applicable laws, ethical guidelines, and institutional review board (IRB) requirements.

---

Citation

If you use ConflLlama-NER in your research, please cite:

@article{meher2025confllama,
  title={ConflLlama: Domain-specific adaptation of large language models for conflict event classification},
  author={Meher, Shreyas and Brandt, Patrick T.},
  journal={Research \& Politics},
  volume={12},
  number={3},
  year={2025},
  publisher={SAGE Publications},
  doi={10.1177/20531680251356282}
}

Note: A separate publication on the NER model is forthcoming. Please check back for updated citation information.

---

Acknowledgments

• Funding: NSF Award 2311142
• Computing Resources: Delta system at NCSA (University of Illinois) through ACCESS allocation CIS220162
• Base Model: Unsloth team for Llama-3.1 8B Instruct optimizations
• Infrastructure: Hugging Face for model hosting and transformers library
• Data Foundation: Global Terrorism Database (GTD) at the University of Maryland

---

License

Apache 2.0 - See LICENSE for details.

---

Related Resources

• ConflLlama Attack Classification: shreyasmeher/confllama
• Research Paper: https://doi.org/10.1177/20531680251356282
• Unsloth Framework: https://github.com/unslothai/unsloth
• CAMEO Coding Scheme: https://eventdata.parusanalytics.com/data.dir/cameo.html

---

Contact

For questions, issues, or collaboration inquiries:

• GitHub Issues: Repository Issues
• Hugging Face: @shreyasmeher

---

<p align="center"> <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/> </p>