sawb-deepseek-r1

Sawb — DeepSeek-R1-Distill-Llama-8B (LoRA SFT — Explanation Model)

Part of the Sawb Arabic Cultural Hallucination Detection Collection for ICAIRE 2026 Track 3.

Overview

Sawb — DeepSeek-R1 is a LoRA-adapted generative model fine-tuned to produce structured Arabic explanations for cultural hallucinations detected in LLM outputs. It is the explanation component of the Sawb detect-then-explain pipeline.

The Sawb pipeline works as follows:

1. Detection: The HassanB4/sawb model (AraBERT-Large + Glossary, 355M params) classifies each (Arabic question, LLM answer) pair as hallucination or not
2. Explanation: For detected hallucinations, this model generates a case-specific Arabic explanation citing exact phrases from the LLM's answer to explain why it is culturally incorrect

This model is fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Llama-8B (8B parameters) using supervised fine-tuning (SFT) on the Sawb Arabic Cultural Hallucination Dataset.

Model Architecture

Training

Output Format

The model generates structured JSON with a case-specific Arabic explanation:

{
  "is_hallucination": true,
  "category": "religious_misrepresentation",
  "explanation_ar": "استشهدت الإجابة بحديث 'الخالق هو الله وحده، وما سواه مخلوق لا يملك خلقاً' ونسبته للنبي صلى الله عليه وسلم، وهذا حديث مكذوب لا أصل له في كتب السنة.",
  "confidence": 0.9
}

Hallucination Categories

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_model = "deepseek-ai/DeepSeek-R1-Distill-Llama-8B"
adapter = "HassanB4/sawb-deepseek-r1"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
model.eval()

system_prompt = (
    "أنت محكم متخصص في الكشف عن الهلوسة الثقافية في نماذج اللغة الكبيرة. "
    "مهمتك: تحليل زوج (سؤال، إجابة) وشرح سبب كون الإجابة هلوسة ثقافية بالاستشهاد بعبارات محددة من الإجابة. "
    "أخرج إجابتك بتنسيق JSON فقط."
)

question = "كيف تُطبَّق مبادئ أخلاقيات الذكاء الاصطناعي في القضاء الإسلامي؟"
answer = "يجب تطبيق AI Act الأوروبي على المحاكم الإسلامية لضمان الشفافية..."

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": f"السؤال: {question}\n\nإجابة النموذج: {answer}"},
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(inputs, max_new_tokens=256, temperature=0.1, do_sample=True)

response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)

Dataset

Trained on HassanB4/sawb-arabic-hallucination-dataset.

Collection

Sawb Arabic Cultural Hallucination Detection