3arabLM-4B-Fiqh-v1

<p align="center"> <img src="pic.jpg" width="400"/> <p> <p align="center"> 💜 <a href="https://github.com/sherif1313/3arabLM/"><b>Github</b></a>&nbsp&nbsp | &nbsp&nbsp🤗 <a href="https://huggingface.co/sherif1313/">Hugging Face</a>&nbsp&nbsp | &nbsp&nbsp📚 <a href="https://github.com/sherif1313/3arabLM">Cookbooks</a>&nbsp&nbsp <br> 🖥️ <a href="https://huggingface.co/spaces/sherif1313/3arabLM-4B-v1">Demo</a>&nbsp&nbsp </a> </p>

🕌 A Specialized Arabic Language Model for Islamic Heritage

*3arabLM is an ongoing research project dedicated to building a large-scale Arabic language model that preserves, memorizes, and reconstructs the classical Islamic scholarly heritage directly from its original sources.

Unlike general-purpose LLMs, this project is not designed to imitate conversations. Its primary objective is the faithful reconstruction of scholarly knowledge while preserving the language, methodology, and diversity of the classical Islamic tradition..*

Research Preview

*This model is an early research and should not be used as a source for religious rulings or legal verdicts. Its purpose is to explore large-scale memorization of the classical Islamic corpus and faithful reconstruction of scholarly texts.

---

5️⃣ Domain Specialization

The model is optimized for:

• 📚 Fiqh (Islamic Jurisprudence)
• 📖 Tafsir (Exegesis)
• The current model represents less than 2% of the planned continual pretraining schedule.
• The full project is expected to expand over multiple stages covering nearly the complete Al-Maktaba Al-Shamela ecosystem.

🌟 Vision

This project aims to build a large-scale Arabic language model primarily trained on Al-Maktaba Al-Shamela and other authoritative Islamic heritage sources.

The objective is to make scholarly knowledge itself part of the model's parameters, rather than relying on external retrieval or internet-scale mixed corpora.

The philosophy behind the project is simple:

• 📖 Learn directly from the original books.
• ✍️ Preserve the language of classical scholars.
• 🎯 Preserve each author's methodology.
• 📚 Preserve scholarly terminology.
• ⚖️ Preserve differences between schools and commentators.

The model is therefore gradually moving toward a paradigm of:

Retrieval from Weights

rather than:

Generative Summarization

Instead of producing heavily paraphrased modern summaries, the model attempts to recall scholarly knowledge from its internal parameters using language close to the original sources.

---

🧭 Project Philosophy

This is not a general conversational model.

Its objective is not creative writing.

Its objective is not producing modern short-form answers.

Instead, the project focuses on:

Faithful reconstruction of classical Islamic knowledge while preserving the original scholarly language and style.

Accordingly, this model can be viewed as a:

• 📖 Knowledge Recall Model
• 🧠 Memorization-Oriented Language Model

rather than simply a:

• 💬 Generative Chat Model

---

✨ What Makes This Model Unique?

1️⃣ Training on Original Sources

Training relies primarily on classical Islamic literature from trusted scholarly sources rather than general web data.

This allows responses to be grounded in the scholarly material the model actually learned instead of relying on broad internet correlations.

2️⃣ Preserving Classical Scholarly Style

The model is trained to maintain traditional Arabic scholarly language.

Typical responses naturally include expressions such as:

• Imam Al-Nawawi said...
• Ibn Kathir said...
• Al-Qurtubi said...
• The majority of scholars held...
• The preferred opinion is...

Instead of modern conversational wording.

3️⃣ Distinguishing Between Scholarly Methodologies

One of the central goals of this project is preventing the collapse of Islamic scholarship into a single homogeneous representation.

The model learns that:

• Tafsir Ibn Kathir is distinct from Tafsir Al-Qurtubi.
• Tafsir Al-Tabari follows its own methodology.
• Every scholar has a unique style of evidence, reasoning, and preference.

Likewise, it learns that:

• A single verse may have multiple valid interpretations.
• A single jurisprudential issue may contain several legitimate opinions.
• Scholarly disagreement is part of the knowledge—not noise to be removed.

4️⃣ Faithful Quotations

Unlike many instruction-tuned language models that aggressively summarize or paraphrase,

this model attempts—whenever possible—to reproduce scholarly explanations in wording close to their original form while preserving technical terminology and classical expressions.

---

🏗️ Long-Term Architecture

The long-term vision is to maintain a stable foundational model and build specialized expert models on top of it.

Examples include:

This architecture allows future domains to be added without retraining the entire base model.

---

📈 Scalability

Future research directions include:

• Copying the final transformer layers with small random perturbations.
• Increasing model capacity through additional layers.
• Preserving previous knowledge while creating space for specialized learning.

The expected advantages are:

• ✅ Better knowledge preservation.
• ✅ Increased memorization capacity.
• ✅ Improved continual training stability.

---

📊 Preliminary Results

Initial experiments indicate that the model is capable of:

• 🧠 Distinguishing between Islamic sciences internally.
• ✍️ Preserving author-specific writing styles.
• 📖 Reproducing scholarly passages with wording close to the originals.
• 🎯 Separating Fiqh, Tafsir, Hadith, and Arabic Language representations during training.

Internal Evaluation

This suggests that the model forms distinct internal representations for different branches of Islamic knowledge rather than treating them as an undifferentiated corpus.

---

🔬 Research Philosophy

The entire project may be summarized by the following statement:

Large Language Models as Compressed Digital Libraries

Rather than viewing language models merely as text generators,

this project explores the possibility of treating them as compressed scholarly libraries capable of faithfully reconstructing classical knowledge from their parameters.

Related research directions include:

• Large Language Models as Compressed Digital Libraries: Full-Parameter Memorization of Classical Arabic Knowledge
• Beyond Retrieval-Augmented Generation: End-to-End Memorization of a Large Arabic Classical Corpus

---

🎯 Design Goal

Unlike conventional instruction-tuned LLMs that tend to summarize or paraphrase,

our objective is faithful knowledge reconstruction from the classical Islamic corpus.

The model is intentionally optimized to reproduce scholarly explanations in their original style rather than generating concise summaries.

---

Why Another Arabic LLM?

Modern large language models are remarkably capable across many domains, yet they are trained on extremely broad internet-scale corpora.

For highly specialized scholarly fields such as classical Islamic sciences, this often introduces several limitations:

Classical references represent only a tiny fraction of the training data. Distinct scholarly methodologies become blended together. Responses are frequently paraphrased instead of preserving original scholarly wording. Source attribution becomes difficult because the model no longer distinguishes where knowledge originated.

3arabLM follows a different philosophy.

Instead of maximizing domain diversity, it maximizes fidelity to a carefully curated corpus of classical Islamic l

Current Capabilities

The current model already demonstrates:

Preservation of classical Arabic scholarly writing. Book-aware text generation. Author-aware stylistic reconstruction. Recognition of differences between jurisprudential schools. Differentiation between major Tafsir methodologies. Long-form scholarly quotations. Reconstruction of passages closely matching the original literature.

---

📦 Current Release

This release represents the first public milestone of a long-term research project.

The model should be viewed as:

An early research preview, not the final objective.

Although the model already demonstrates promising capabilities in preserving classical Arabic scholarly language and recalling information from Islamic heritage, it has been trained on less than 2% of the planned training curriculum.

The long-term vision extends far beyond this initial release.

---

📚 Training Corpus

The foundation of this project is Al-Maktaba Al-Shamela, one of the largest digital collections of classical Islamic literature.

Official Library: goldenshamela

The original Shamela library contains approximately 35,000 classical Arabic books covering nearly every branch of Islamic scholarship, including:

• 📖 Tafsir
• 📜 Hadith
• ⚖️ Fiqh
• 🕌 Aqeedah
• 📝 Usul al-Fiqh
• 📚 Arabic Grammar
• 🏛 History
• 👤 Biography
• 📝 Literature
• 📖 Lexicons
• And many other disciplines.

Rather than using these books directly as plain text, the entire corpus is reconstructed through a dedicated extraction pipeline designed specifically for language-model training.

---

🔧 Knowledge Extraction Pipeline

The corpus is transformed into a structured representation through multiple stages.

                    Shamela
                       │
        ┌──────────────┴──────────────┐
        │                             │
      Books                       Indices
        │                             │
        ▼                             ▼
   Text Parser                  Keyword Parser
        │                             │
        └──────────────┬──────────────┘
                       ▼
            Unified Page Builder
                       │
      ┌────────────┬────────────┬────────────┐
      ▼            ▼            ▼
   Metadata        TOC       Keywords
                       │
                       ▼
                 pages.parquet
                       │
      ┌────────────┬────────────┬────────────┐
      ▼            ▼            ▼
      CPT          SFT          RAG
                       │
        Book → Section → Paragraph → Sentence

This pipeline preserves the hierarchical structure of the original books instead of flattening them into raw text.

Consequently, the model learns not only the words, but also the relationships between books, chapters, sections, and scholarly discussions.

---

🌐 Why Not Train on the General Internet?

Most modern Large Language Models are trained on extremely broad internet-scale corpora.

While this makes them highly versatile, it also introduces significant challenges for specialized scholarly domains.

For example:

• Classical Islamic sources become a very small fraction of the overall training data.
• The model often loses awareness of the original source of a statement.
• Distinct scholarly methodologies become mixed together.
• Modern paraphrases may dominate over the original classical wording.

This project follows a different philosophy.

Instead of maximizing domain diversity, it maximizes scholarly consistency within a carefully curated corpus.

---

🗺️ Long-Term Research Strategy

The long-term architecture is based on a single foundational model followed by multiple specialized expert models.

Instead of independently training several large models from scratch, the project follows a staged approach.

Stage 1 — Foundation Model

The base model is continually pretrained on a balanced mixture of Islamic sciences to develop a unified understanding of classical Arabic.

This foundation becomes the permanent reference model for the project.

Stage 2 — Specialized Experts

Each expert inherits the linguistic and scholarly knowledge of the foundation model while specializing in its own discipline.

This architecture allows new experts to be added without retraining the entire system.

---

🔮 Future Dataset Evolution

Future releases will gradually enrich the training data with structured metadata instead of relying solely on plain text.

Planned metadata includes:

The objective is to strengthen the model's internal understanding of scholarly context rather than treating every passage as isolated text.

---

🧪 Future Research Directions

The project is currently exploring several research ideas, including:

• 🧠 Metadata-aware continual pretraining.
• 📈 Knowledge-oriented transformer scaling.
• 🔧 Layer expansion through duplicated upper transformer blocks with small stochastic initialization.
• 💾 Higher-capacity memorization without catastrophic forgetting.
• 🎯 Expert-specialized adapters built upon a single foundational model.

These ideas aim to transform the model from a general language model into a faithful digital representation of the classical Islamic scholarly tradition.

---

🎯 Project Goal

The ultimate objective is not to replace scholars.

Nor is it simply to build another conversational assistant.

The objective is to develop a language model capable of:

Preserving, recalling, and reconstructing the classical Islamic heritage with the highest possible degree of fidelity while maintaining the language, methodology, and scholarly diversity found in the original sources.

---

🗓️ Future Roadmap

This release represents only the beginning of the project.

The model has been trained on a small fraction of the planned corpus, and future releases will substantially expand both the breadth and depth of the model's knowledge.

The long-term objective is to build the most comprehensive Arabic language model dedicated to the Islamic scholarly heritage.

---

📖 Planned Corpus Expansion

---

📚 Towards a Comprehensive Digital Library

The ultimate objective is to transform the model into a compressed digital representation of the Islamic scholarly tradition.

Rather than specializing in only one branch of knowledge, the long-term model will gradually encompass nearly every major discipline preserved within Al-Maktaba Al-Shamela.

Each training stage will improve:

• 📈 Breadth of scholarly coverage.
• 📊 Depth of domain knowledge.
• ✍️ Preservation of author-specific writing styles.
• 🎯 Recognition of methodological differences between scholars.
• 📖 Faithful reconstruction of classical texts.

As the corpus grows, the model will increasingly function as an internal scholarly library whose knowledge is stored directly within its parameters rather than reconstructed from fragmented internet sources.

This roadmap represents the long-term vision of the project: a continually evolving foundation model upon which increasingly specialized expert models can be built while preserving a unified understanding of the classical Islamic heritage.

---

🚀 Usage Example The code must be used because it is compatible with the training method.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

BASE_MODEL = "sherif1313/3arabLM-4B-Fiqh-v1"

print("⏳ تحميل النموذج...")
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, 
    dtype=torch.bfloat16, 
    device_map="auto", 
    trust_remote_code=True
)
model.eval()

def generate_text(prompt, book="التفسير", author="ابن كثير", category="التفاسير"):
    # الالتزام بتوزيع التدريب
    full_prompt = f"book {book}\nauthor {author}\ncategory {category}\n{prompt}"
    
    print(f"\n📝 السؤال/النص: {prompt}")
    print("-" * 50)
    
    inputs = tokenizer(full_prompt, return_tensors="pt").to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            do_sample=True,
            temperature=0.15,          # إعداداتك الممتازة
            top_p=0.85,
            top_k=40,
            repetition_penalty=1.08,
            no_repeat_ngram_size=5,
            max_new_tokens=300,        # تم رفع الحد قليلاً لتناسب الأسئلة الفقهية
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.eos_token_id,
        )
    
    answer = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
    print(f"🤖 إجابة النموذج:\n{answer}")
    print("=" * 80)

# قائمة بالأسئلة المتنوعة لاختبار النموذج
test_cases = test_cases = [
    # ==================== 1. التفسير ====================
    {
        "prompt": "وقوله تعالى: {إِيَّاكَ نَعْبُدُ وَإِيَّاكَ نَسْتَعِينُ} أي:",
        "book": "تفسير ابن كثير",
        "author": "ابن كثير",
        "category": "التفاسير"
    },
    {
        "prompt": "فسر قوله تعالى: {الَّذِينَ آمَنُوا وَلَمْ يَلْبِسُوا إِيمَانَهُمْ بِظُلْمٍ} [الأنعام: 82]",
        "book": "تفسير الطبري",
        "author": "الطبري",
        "category": "التفاسير"
    },
    {
        "prompt": "ما هو سبب نزول سورة الفاتحة؟",
        "book": "تفسير القرطبي",
        "author": "القرطبي",
        "category": "التفاسير"
    },
    {
        "prompt": "ماذا قال المفسرون في قوله تعالى: {وَإِنْ تَعُدُّوا نِعْمَةَ اللَّهِ لا تُحْصُوهَا} [النحل: 18]؟",
        "book": "تفسير الرازي",
        "author": "الرازي",
        "category": "التفاسير"
    },
    {
        "prompt": "ما معنى الاستعاذة في قوله تعالى: {فَاسْتَعِذْ بِاللَّهِ إِنَّهُ هُوَ السَّمِيعُ الْبَصِيرُ} [غافر: 56]؟",
        "book": "تفسير البغوي",
        "author": "البغوي",
        "category": "التفاسير"
    },

    # ==================== 2. الفقه (العبادات) ====================
    {
        "prompt": "ما حكم المسح على الخفين؟",
        "book": "الأم",
        "author": "الشافعي",
        "category": "فقه عام"
    },
    {
        "prompt": "ما هي شروط صحة الصلاة؟",
        "book": "المجموع",
        "author": "النووي",
        "category": "فقه عام"
    },
    {
        "prompt": "ما حكم ترك الصلاة؟",
        "book": "المغني",
        "author": "ابن قدامة",
        "category": "فقه عام"
    },
    {
        "prompt": "كم عدد ركعات صلاة الجمعة؟",
        "book": "بدائع الصنائع",
        "author": "الكاساني",
        "category": "فقه عام"
    },
    {
        "prompt": "ما هي مبطلات الوضوء؟",
        "book": "الهداية",
        "author": "المرغيناني",
        "category": "فقه عام"
    },
    {
        "prompt": "ما حكم صلاة المسافر؟",
        "book": "المجموع",
        "author": "النووي",
        "category": "فقه عام"
    },
    {
        "prompt": "هل يجوز الجمع بين الصلاتين في السفر؟",
        "book": "المغني",
        "author": "ابن قدامة",
        "category": "فقه عام"
    },

    # ==================== 3. الفقه (المعاملات) ====================
    {
        "prompt": "ما حكم بيع الرطب بالتمر؟",
        "book": "الأم",
        "author": "الشافعي",
        "category": "فقه عام"
    },
    {
        "prompt": "ما حكم بيع الذهب بالذهب؟",
        "book": "المبسوط",
        "author": "السرخسي",
        "category": "فقه عام"
    },
    {
        "prompt": "ما هي شروط صحة عقد البيع؟",
        "book": "المجموع",
        "author": "النووي",
        "category": "فقه عام"
    },
    {
        "prompt": "ما حكم الربا في الإسلام؟",
        "book": "المغني",
        "author": "ابن قدامة",
        "category": "فقه عام"
    },
    {
        "prompt": "هل يجوز بيع الدين بالدين؟",
        "book": "بدائع الصنائع",
        "author": "الكاساني",
        "category": "فقه عام"
    },
    {
        "prompt": "ما حكم الغش في البيع؟",
        "book": "الأم",
        "author": "الشافعي",
        "category": "فقه عام"
    },

    # ==================== 4. الفقه (الزواج والطلاق) ====================
    {
        "prompt": "ما هي شروط صحة عقد النكاح؟",
        "book": "المجموع",
        "author": "النووي",
        "category": "فقه عام"
    },
    {
        "prompt": "ما حكم الطلاق في الإسلام؟",
        "book": "المغني",
        "author": "ابن قدامة",
        "category": "فقه عام"
    },
    {
        "prompt": "كم عدد الطلقات التي يملكها الزوج؟",
        "book": "الهداية",
        "author": "المرغيناني",
        "category": "فقه عام"
    },

    # ==================== 5. الفقه (الجنائز) ====================
    {
        "prompt": "ما هي أحكام غسل الميت؟",
        "book": "المجموع",
        "author": "النووي",
        "category": "فقه عام"
    },
    {
        "prompt": "ما حكم الصلاة على الميت؟",
        "book": "المغني",
        "author": "ابن قدامة",
        "category": "فقه عام"
    },

    # ==================== 6. الفقه (الجهاد) ====================
    {
        "prompt": "ما حكم الجهاد في الإسلام؟",
        "book": "المبسوط",
        "author": "السرخسي",
        "category": "فقه عام"
    }
]

print("\n🚀 بدء اختبار النموذج على أسئلة متنوعة...\n")

for test in test_cases:
    generate_text(test["prompt"], test["book"], test["author"], test["category"])

Current Limitations

This release should be considered an early research .

Current limitations include:

Less than 2% of the planned continual pretraining has been completed. Some domains remain significantly underrepresented. The model may occasionally mix neighboring passages. Citation grounding is still under active development. Long-context reconstruction will improve in future releases.

This project explores several research directions:

Full-parameter memorization of a classical corpus. Metadata-aware continual pretraining. Hierarchical book representation. Weight-based knowledge retrieval. Capacity scaling through layer expansion. Domain-specialized expert models.

Why Memorization?

Most recent research focuses on Retrieval-Augmented Generation (RAG), where knowledge remains outside the model and is retrieved at inference time.

This project investigates the opposite direction:

Can a large language model become a compressed scholarly library whose knowledge is stored directly in its parameters?

Our hypothesis is that sufficiently large continual pretraining on a carefully curated corpus allows faithful reconstruction of classical scholarly knowledge without relying on external retrieval for every query.

🤝 Final Note

This project seeks to preserve the scientific heritage of the Islamic library inside a language model—faithfully, transparently, and with respect for the diversity of classical scholarship.

If you have any questions, would like to contribute, or have ideas for future research directions, please feel free to reach out.

The journey is just beginning.

📎 Citation

If you use this model in your research, please cite:

@misc{shamela-arabic-foundation, author = {Sherif1313}, title = {3arabLM-4B-Fiqh-v1: A Specialized Arabic Language Model for Islamic Heritage}, year = {2026}, publisher = {Hugging Face}, url = { https://huggingface.co/sherif1313/3arabLM-4B-Fiqh-v1 } }