IndoVSE

IndoVSE — Indonesian Video Salient Entity Detection

IndoVSE is the first model for salient named entity detection in Indonesian news videos. Given a raw video file, the pipeline automatically identifies which named entities are topically central (salient) versus incidentally mentioned.

Installation

pip install indovse

Note: FFmpeg must be installed on your system.

• Linux: apt install ffmpeg
• Mac: brew install ffmpeg
• Windows: ffmpeg.org/download.html

All models (~3GB) are downloaded automatically on first use and cached locally.

Quick Start

from indovse import predict

result = predict("berita.mp4")

print(result["salient_entities"])
# [
#   {"entity_text": "Prabowo Subianto", "entity_label": "PER", "salient_prob": 0.874, ...},
#   {"entity_text": "KPU",             "entity_label": "ORG", "salient_prob": 0.811, ...},
#   {"entity_text": "Pemilu 2024",     "entity_label": "EVT", "salient_prob": 0.743, ...},
# ]

print(result["entity_timeline"])
# {
#   "Prabowo Subianto": [2.3, 45.1, 102.5, 189.0],
#   "KPU": [5.2, 78.9, 134.2],
# }

Output Format

{
  "video_duration": 312.5,        # second
  "total_entities": 18,           # total entity
  "salient_entities": [
    {
      "entity_text":  "Prabowo Subianto",
      "entity_label": "PER",
      "salient_prob": 0.874,      # salient probability [0-1]
      "freq_norm":    0.910,      # appearance frequency (normalized)
      "first_appear": 0.020,      # first appear position (normalized)
      "coverage":     0.840,      # spread (normalized)
      "burstiness":   0.310       # spread pattern (-1 tersebar, +1 menumpuk)
    }
  ],
  "entity_timeline": {
    "Prabowo Subianto": [2.3, 45.1, 102.5, 189.0]  # timestamp in second
  }
}

Parameters

predict(
    video_path,   # path to video file (.mp4, .mkv, .webm, dll)
    top_k=5       # maximum salient entity return
)

How It Works

Video
  → ffmpeg               (ekstrak audio)
  → Whisper large        (speech-to-text + word timestamps)
  → IndoBERT NER         (named entity recognition)
  → Temporal features    (freq_norm, burstiness, first_appear, coverage)
  → IndoBERT CLS         (contextual embedding)
  → IndoVSE MLP          (salience classification)
  → Ranked salient entities

Temporal Features

Performance

Ablation Study

Dataset

Trained on IndoVSE-dataset — 573 Indonesian news videos (~8.6 hours) from Metro TV, Kompas TV, and tvOne covering 4 domains: politik, ekonomi, kesehatan, pendidikan.

• 27,115 entity instances
• 2,193 salient (8.1%) / 24,922 non-salient (91.9%)
• Human-annotated, inter-annotator agreement κ = 0.931

Models Used

Citation

@misc{indovse2025,
  title     = {IndoVSE: Temporal Entity Salience Detection for Indonesian News Videos},
  author    = {Galih Kusuma Wijaya},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/galihkjaya/IndoVSE}
}

License

MIT