Downloads · 30 days
4
15% of all-time downloads
alirezaaminzadeh/petrosafe-rag-retriever
petrosafe-rag-retriever is a machine learning model from alirezaaminzadeh. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for scikit-learn. The card lists the license as apache-2.0.
Hybrid BM25 + word TF-IDF + character TF-IDF lexical retrieval index over the bilingual (Persian/English) PetroSafe process-safety corpus. CPU-only — not a fine-tuned transformer, no GPU required to build or to serve.
Downloads · 30 days
4
15% of all-time downloads
All-time downloads
26
Public
Repo size
1 MB
Likes
0
Public
Click a slice to open those files.
.joblib1 MB · 99%
From the Hugging Face model README
Hybrid BM25 + word TF-IDF + character TF-IDF lexical retrieval index over the bilingual (Persian/English) PetroSafe process-safety corpus. CPU-only — not a fine-tuned transformer, no GPU required to build or to serve.
Architecture is unchanged from hse-multimodal-rag-retriever:
a word-level TF-IDF (1-2 grams), a character TF-IDF (char_wb 3-5 grams, robust to Persian
morphology and typos), and an Okapi BM25 index, combined as 0.42*word + 0.28*char + 0.30*bm25.
Query-side keyword overlap adds a small additional boost. This project only supplies new fitted
weights over new (oil/gas HSE) bilingual content — see the companion dataset repo for the corpus.
| File | Description |
|---|---|
hybrid_index.joblib | Fitted TfidfVectorizer (word + char) + BM25 index + chunk-id order |
config.json | Index stats (backend, chunk count, vocab sizes) |
eval_results.json | Retrieval/abstention metrics for this exact index (see dataset card for detail) |
{
"backend": "hybrid_tfidf_bm25",
"n_chunks": 56,
"word_vocab": 4341,
"char_vocab": 14403
}
scripts/evaluate.py)| Metric | Value |
|---|---|
| Retrieval doc-hit rate | 96.15% |
| Keyword hit rate | 96.15% |
| Correct-abstention rate | 100% (out-of-scope AND unknown-equipment-tag questions) |
If a question names an equipment-tag-shaped token (e.g. V-101) that has zero matching chunks
in the corpus, the retriever's metadata filter excludes every candidate and returns no hits — the
generator then abstains automatically rather than answering from an unrelated chunk. This is
exercised explicitly in the evaluation set (4 of the 8 negative QA pairs use a plausible-looking
but non-existent tag such as V-999, X-888, K-777, R-999).
Companion dataset: alirezaaminzadeh/petrosafe-rag-corpus-fa
Space: alirezaaminzadeh/petrosafe-rag-fa