tax-retriever-bge-small

tax-retriever-bge-small

A retrieval embedding model fine-tuned for Indian income-tax question answering. Built on BAAI/bge-small-en-v1.5 and adapted to the Income-Tax Act 2025 and professional tax-guide corpus used by the RAG Tax Assistant application.

Model

Training data

The corpus contains 776 documents:

• 629 sections of the Income-Tax Act 2025 (machine-readable text, from the official Act consolidated text)
• 147 professional tax guides covering deductions (80C/80D/80E/80G), HRA, capital gains, ITR forms, NRI taxation, TDS and related topics

Training pairs are self-supervised: each section's title and heading are paired with its body text, so the model learns to rank the correct provision for a tax-shaped query. Word embeddings were frozen during training to preserve the base vocabulary while adapting the encoder.

Evaluation

Real-query retrieval (pilot)

20 user-style questions (not present in training pairs) were retrieved against the full corpus; a hit means the top-ranked document belongs to the question's topic family.

Pilot scale (n=20): the 0.85 vs 0.80 margin is one question and is not statistically significant (Clopper-Pearson CIs overlap; McNemar on n=20 cannot resolve a 1-question flip). Reported as directionally consistent, not as a measured win. A larger labeled set is the planned next step.

Section retrieval (exact match)

Leave-one-out over all 776 documents: the query is a document's title and the target is its own content. Both models evaluated on the identical protocol and corpus (controlled comparison; full results in eval/tax_retriever_fixed_eval.json). This metric is close to the training objective (title→content pairs), so it reads as a training-recall check rather than a generalization claim.

Baseline sanity check

A TF-IDF retriever was evaluated under the topic-family protocol as a non-neural control. It scores 0.91 hits@1 on the family metric, which is expected: the family protocol rewards lexical overlap between near-identical Act sections. The exact-match and real-query protocols above do not share this property and are the reported metrics.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("vivekkopthsd/tax-retriever-bge-small")

# BGE models require the query prefix on the query side only
queries = ["Represent this sentence for searching relevant passages: "
           "How much can I deduct under Section 80C?"]
docs = ["Represent this sentence for searching relevant passages: "
        "Section 80C of the Income-Tax Act, 2025."]

q_vec = model.encode(queries, normalize_embeddings=True)
d_vec = model.encode(docs, normalize_embeddings=True)

Limitations

• Trained on Indian income-tax texts; performance on other legal or financial domains is not evaluated.
• The training corpus is English; Hindi queries are encoded by an English-only tokenizer.
• Section retrieval is measured on the training corpus; cross-corpus generalization is not yet measured.
• Not legal advice. Provisions change with each Finance Act; verify against the official consolidated Act text.

Training

Reproducible training notebook: Kaggle — tax-retriever-finetune-bge-small

Training corpus dataset: India Tax Corpus