egis-emer-guard

EGIS (Full) — EMER-Guard

Research checkpoint for EGIS (Full), the evidence-conditioned multi-task verifier from:

EMER-Guard: Protocol-Grounded Verification for LLM Emergency Instructions (EMNLP 2026 Main).

Code and data archives (software.tgz + data.tgz): https://huggingface.co/datasets/qq0518/emer-guardbench.

This is the egis_v2_clean_splitfix/best_model.pt file used for the reported main-table numbers. It is not a Hugging Face AutoModel checkpoint: load it with the released MultiTaskEGIS class, then load_state_dict.

Intended use and limits

EGIS is a controlled, mechanism-level scientific probe, not a deployment-ready emergency verifier. It was trained and evaluated on EMER-GuardBench (counterfactual edits of curated FEMA/CDC/WHO/Red Cross protocols). Do not use these weights as a substitute for incident-specific professional judgment, local authority guidance, or real-world alert evaluation.

Files

Architecture

• Backbone: BAAI/bge-m3 (XLM-RoBERTa, hidden 1024, 24 layers).
• Parameter count in this state_dict: 568,289,806 (paper: 568.3M). 152.7M of those were fine-tuned; embeddings and the lower half of the encoder were frozen at train time.
• Heads (keys in the state_dict):
  • hazard_head: 2-way safe / hazardous
  • mechanism_head: MLP, 8 logits (m1–m8 as in the released training code)
  • severity_head: scalar
  • support_head: 3-way supported / unsupported / contradicted
• Input format: [Evidence] {passage} [SEP] {instruction}, max_length=256.
• Decision threshold for the reported supervised numbers: hazard probability 0.5.

How to load

hf download qq0518/egis-emer-guard --local-dir ./egis_ckpt

Then, with the EMER-Guard software archive on PYTHONPATH:

import torch
from transformers import AutoModel, AutoTokenizer
from train_egis import MultiTaskEGIS  # from the released software/src/train_egis.py

encoder = "BAAI/bge-m3"  # or a local BGE-M3 directory
tok = AutoTokenizer.from_pretrained(encoder)
model = MultiTaskEGIS(encoder, local_files_only=False)
sd = torch.load("egis_ckpt/best_model.pt", map_location="cpu", weights_only=False)
model.load_state_dict(sd, strict=True)
model.eval()

One-command evaluation (software archive + sibling data/ tree):

export EMERGUARD_ENCODER_PATH=/path/to/bge-m3   # optional local encoder
./reproduce.sh --model_path ./egis_ckpt/best_model.pt

Initializing MultiTaskEGIS still downloads (or reads a local copy of) the base BGE-M3 architecture; the fine-tuned weights come from best_model.pt.

Reported IID numbers (from test_results.json)

These are the metrics written beside this checkpoint after the original test eval. They match the paper's EGIS (Full) headline operating point (Harm F1 89.9, IID ORR 10.4%). Benign-stress 0/1,000 false positives is reported in the paper, not in this JSON (that eval used a separate CSV).

Training (original run)

From train_config.json: AdamW, 3 epochs, batch 4 × 8 grad accumulation, lr 2e-5, bfloat16, gradient checkpointing, seed 42, lam_mech=1.5, label smoothing 0.05, evidence on, 30,000 counterfactual pairs. Hardware reported in the paper: two NVIDIA RTX 4090 D GPUs, ~12 GPU-hours.

License

Apache 2.0 for these fine-tuned weights and metadata. The BGE-M3 base encoder is from BAAI (BAAI/bge-m3).