Bible-Assistant-Qwen3.5-4B-v2

Superseded (2026-09-05). Bible-Assistant-Qwen3.5-4B-v3.2 is the current model -- 80.3% verse-quote exact / 98.9% citation / 1.9% hallucination, best in its size class on every metric tested. This v2 checkpoint is kept for reproducibility and the project's version history (see the v3.2 card's "Version history" section), not as the recommended download.

Bible AI Assistant — v2-4b (Qwen3.5-4B SFT)

A locally-runnable, retrieval-grounded Bible Q&A model fine-tuned on Qwen3.5-4B. Built for a single 16 GB consumer GPU. On its own protocol-v3 benchmark (282 questions, verse-level verification) it reaches 76.5 % verbatim verse recall on direct verse-lookup questions, 98.9 % citation rate, and ~2 % hallucination — and it is trained to decline rather than counsel on crisis / pastoral questions. (Exact-verse accuracy across all verse-scored categories is 29.3 %; see Evaluation for why the synthesis categories score 0 on that metric.)

Scope. This is the supervised-fine-tuning stage of a multi-stage pipeline. It is strong on the core retrieval task and on safe refusal, and currently weaker at open-ended thematic synthesis ("explain the context of Psalm 23"), where it tends to list verses rather than explain — a known limitation of the template-heavy training data, addressed in the planned v3 (teacher-distilled answers + a GRPO faithfulness stage). Full numbers and a controlled A/B vs. the prior model are in Evaluation.

• Project & RAG pipeline: https://github.com/t-timms/bible-ai-assistant
• GGUF quants: Ttimms/Bible-Assistant-Qwen3.5-4B-v2-GGUF

Architecture

graph TD
    Base["Qwen/Qwen3.5-4B<br/>base model"]

    subgraph Build ["Build pipeline - RTX 5070 Ti"]
        SFT["bf16 LoRA SFT - 1 epoch<br/>56k-example v2 dataset (8 categories)"]
        Merge["Merge adapter -> bf16"]
        Conv["convert_hf_to_gguf --no-mtp<br/>+ llama-quantize (+ imatrix)"]
    end

    subgraph HF ["Published on Hugging Face"]
        ST["Bible-Assistant-Qwen3.5-4B-v2<br/>safetensors"]
        GG["...-v2-GGUF<br/>Q4_K_M / Q5_K_M / Q6_K / Q8_0 / F16 (+imatrix)"]
    end

    subgraph Serve ["Retrieval-augmented serving"]
        RAG["RAG server (FastAPI)<br/>dense (nomic-embed) + BM25 + RRF + bge-reranker-v2-m3"]
        LLM["Ollama / llama.cpp<br/>Qwen3.5-4B SFT"]
    end

    Base --> SFT --> Merge --> ST
    Merge --> Conv --> GG
    ST --> RAG --> LLM
    GG --> LLM

Summary

What it's for

A conversational model for answering Bible questions as part of a RAG pipeline: a retriever fetches relevant passages and passes them as context; the model answers from that context. It is not designed to be used context-free.

Appropriate: personal Bible study, verse lookup, sermon-prep passage finding, devotional Q&A, educational exploration — with retrieval running.

Not for: medical / legal / financial advice; counselling or pastoral care (the model is trained to redirect these to a pastor or crisis line); authoritative theological decisions; unsupervised or at-scale deployment. Not adversarially red-teamed.

Training data

Produced by the project's v2 dataset engine — 56,022 examples, fully provenance-tracked (per-source SHA + license in the manifest), and decontaminated against every question in the evaluation suite (zero overlap).

No proprietary, personal, or commercially licensed data.

Training

LoRA (r=32, α=64, dropout 0.05) on q/k/v/o/gate/up/down proj, bf16 (fully unquantized), 1 epoch / 3,474 steps, effective batch 16, lr 2e-4 cosine, seq 1280 fixed-pad, completion-only loss masking. ~10.4 h on a single RTX 5070 Ti (16 GB). Eval loss 0.2515 → 0.2138, monotonic over all 70 evals — no overfitting.

Evaluation

Benchmark protocol v3 (282 questions, sha-pinned suite), keyword/verification metrics, greedy decode, seed 42, RAG context enabled.

Row counts sum to the 266 verse-scored questions; the remaining 16 of the 282 are refusal / pastoral-triage questions, scored separately for decline/redirect correctness and excluded from the rate denominators above.

* verse_accuracy scores "quoted the one expected verse verbatim." Character / topical / context / theological questions have no single canonical verse answer, so a good synthesised answer scores 0 on this metric — the fuzzy column and a judge pass score them fairly.

Versus the previously shipped model (same protocol, same day): verse-lookup exact accuracy 58 % → 76.5 % (+18.5 pp), citation rate 88 % → 98.9 % (+11 pp), hallucination flat at ~2 %. But overall fuzzy mean regressed 0.48 → 0.40 — the lightly-tuned prior model's thematic answers are closer to the expected natural answers. Diagnosis: the dataset's rigid fill-in-the-blank answer templates taught this model the format rather than the skill, so open-ended "explain / who is / what is the context of" questions get a verse list instead of an explanation. This is the target of the planned v3 (teacher-distilled answers + GRPO).

Limitations

• Thematic-answer regression (above) — dataset issue, targeted for v3.
• RAG dependency — reliable verse accuracy needs the retriever + index; without context the model falls back to parametric memory. Always verify cited verses against a Bible.
• Ollama not yet — GGUF quants are provided and run in current llama.cpp / recent LM Studio, but Ollama 0.33.x's bundled runtime is too old for the qwen35 arch. Use once Ollama updates, or run llama.cpp directly.
• Sequence length — trained at 1280 tokens; longer inputs truncate.
• English only.
• SFT-only — the preference and RL stages the pipeline is designed around have not run.

Bias

Protestant canon across six public-domain English translations (no Deuterocanonical books); Matthew Henry's commentary reflects an 18th-century Reformed Protestant perspective. The pastoral_triage data deliberately models tradition-aware framing ("faithful Christians differ on …") for disputed questions, but the sources still lean evangelical / Reformed Protestant. Inherits any biases in Qwen3.5-4B. Hand-authored escalation data reflects the developer's judgment. Apply critical judgment, especially on contested theological questions.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

m = "Ttimms/Bible-Assistant-Qwen3.5-4B-v2"
tok = AutoTokenizer.from_pretrained(m, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.bfloat16, device_map="cuda", trust_remote_code=True)

# The model expects a retrieval-augmented prompt: verses in a Context block, then the question.
user = (
    "Context:\n- **John 3:16**: For God so loved the world, that he gave his only begotten "
    "Son, that whosoever believeth in him should not perish, but have everlasting life.\n\n"
    "Q: What does John 3:16 say?"
)
msgs = [
    {"role": "system", "content": "You are a Bible AI assistant. Answer questions about Scripture accurately and conversationally."},
    {"role": "user", "content": user},
]
text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False, enable_thinking=False)
ids = tok(text, return_tensors="pt").input_ids.to("cuda")
out = model.generate(ids, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

The full RAG server (retriever + reranker + citation verification) is in the project repo.

License

• Weights: Apache-2.0 (from Qwen3.5-4B).
• Code: MIT (project repo).
• Bible translations: public domain. Matthew Henry's Commentary: CC0.
• smoltalk2: Apache-2.0 for its new subsets; inherited subsets keep upstream licenses.

Citation

@misc{bible-ai-assistant-2026,
  title        = {Bible AI Assistant: A RAG-Grounded Bible Q\&A Model Fine-tuned on Qwen3.5-4B},
  author       = {Tremayne Timms},
  year         = {2026},
  howpublished = {GitHub},
  url          = {https://github.com/t-timms/bible-ai-assistant}
}