LF_BERT_v1

LF_BERT_v1

LF_BERT_v1 is a lightweight TinyBERT-based cross-encoder fine-tuned for semantic evidence filtering in Retrieval-Augmented Generation (RAG) pipelines.

The model acts as a semantic gatekeeper, scoring (query, candidate_sentence) pairs to determine whether the sentence is factually useful evidence or a semantic distractor.
It is designed for CPU-only, edge, and offline deployments, with millisecond-level inference latency.

This model is the core filtering component of Project Sentinel.

---

Model Description

• Architecture: TinyBERT (4 layers, 312 hidden size)
• Type: Cross-encoder (joint encoding of query and sentence)
• Task: Binary fact-checking / evidence verification
• Base Model: huawei-noah/TinyBERT_General_4L_312D
• Inference Latency: ~5.3 ms (CPU)

Input Format

[CLS] query [SEP] candidate_sentence [SEP]

• Maximum sequence length: 512 tokens

Output

• Probability score ∈ [0,1] representing factual utility
• Typical deployment threshold: 0.85 (Strict Guard configuration)

---

Intended Use

✔ Semantic filtering for RAG pipelines
✔ Hallucination reduction
✔ Early-exit decision systems
✔ Edge / offline LLM deployments

This model is especially suited for:

• Local document QA systems
• Privacy-sensitive environments
• Resource-constrained hardware (≤ 8 GB RAM)

---

Limitations

• Trained on Wikipedia-based QA (HotpotQA)
• English-only
• Sentence-level relevance (not passage-level reasoning)
• Not a factual verifier for open-world claims

Performance may degrade on highly domain-specific or non-factual corpora.

---

Training Data

The model was trained on a binary dataset derived from HotpotQA (Distractor setting).

Labels

• 1 – Supporting Fact: Ground-truth evidence sentences
• 0 – Distractor: Topically similar but factually insufficient sentences

Dataset Statistics

The dataset is intentionally imbalanced, reflecting real retrieval scenarios.

---

Training Procedure

Hyperparameters

• Learning rate: 1e-5
• Batch size: 16
• Epochs: 2
• Optimizer: AdamW
• Scheduler: Linear
• Seed: 42
• Loss: Weighted cross-entropy

Training Results

---

Thresholded Performance (Strict Guard)

• Decision threshold: 0.85
• Hallucination rate: 5.92%
• Fact retention: 60.34%
• Average latency: 5.30 ms (CPU)

This configuration prioritizes trustworthiness over recall.

---

Citation

If you use this model, please cite:

@article{salih2026sentinel,
  title={Project Sentinel: Lightweight Semantic Filtering for Edge RAG},
  author={Salih, El Mehdi and Ait El Mouden, Khaoula and Akchouch, Abdelhakim},
  year={2026}
}

---

Contact

El Mehdi Salih
Mohammed V University – Rabat
Email: [email protected]