Downloads · 30 days
16
17% of all-time downloads
vivekkopthsd/minilm-l6-v2-taxfinance-lora
minilm-l6-v2-taxfinance-lora is a sentence similarity model from vivekkopthsd. Use it when you need a score for how close two texts are. It is set up for sentence-transformers. The card lists the license as apache-2.0.
A parameter-efficient retrieval encoder fine-tuned from sentence-transformers/all-MiniLM-L6-v2 for Indian tax and finance question-answering retrieval.
Downloads · 30 days
16
17% of all-time downloads
All-time downloads
93
Public
Parameters
22.7M
90.9 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors90.9 MB · 99%
From the Hugging Face model README
A parameter-efficient retrieval encoder fine-tuned from
sentence-transformers/all-MiniLM-L6-v2
for Indian tax and finance question-answering retrieval.
The model maps a taxpayer's question to the exact statute passage (or financial document) it is answered by. It is compact (22M params, ~90 MB), fast to embed, and cheap to deploy — designed as a retrieval unit for a RAG pipeline on modest hardware.
| Property | Value |
|---|---|
| Base model | all-MiniLM-L6-v2 (22.7M params) |
| Fine-tuning | LoRA (r=16, α=32, dropout 0.05) on query/key/value/dense |
| Trainable params | 675,840 — 2.89% of the model |
| Embedding dim | 384 |
| Max sequence length | 256 tokens |
| Loss | MultipleNegativesRankingLoss |
| Compute | GPU (Kaggle), P100 |
| License | Apache-2.0 (base model) |
Measured on a held-out pooled retrieval test of 839 queries scored against the full combined tax + finance corpus (100% unseen during training). Reported as mean ± std across 3 independent training seeds (42, 123, 456).
| Metric | LoRA fine-tuned | Zero-shot baseline | Δ |
|---|---|---|---|
| nDCG@10 | 0.592 ± 0.001 | 0.447 | +0.145 |
| MRR@10 | 0.484 ± 0.002 | 0.361 | +0.123 |
| MAP@100 | 0.488 ± 0.002 | 0.373 | +0.115 |
| Recall@1 | 0.327 ± 0.002 | 0.250 | +0.076 |
| Recall@5 | 0.685 ± 0.001 | 0.496 | +0.189 |
| Recall@10 | 0.948 ± 0.002 | 0.735 | +0.212 |
| Accuracy@1 | 0.327 ± 0.002 | 0.250 | +0.076 |
Fine-tuning delivers a +32% relative improvement in nDCG@10 over the untrained base, with high seed-to-seed stability (std ≤ 0.002).
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("vivekkopthsd/minilm-l6-v2-taxfinance-lora")
query = model.encode(
["What is the deduction for rent paid on salary income?"],
normalize_embeddings=True,
)
docs = model.encode(
[
"Section 134: Deduction in respect of rent paid.",
"Section 123: Deduction for life insurance premium.",
],
normalize_embeddings=True,
)
scores = query @ docs.T
Data — pooled from two sources:
ligaments-dev/indian-income-tax-qa — 6,800 (question, answer) pairs from
Indian Income-Tax statutes, used as (query, positive-document) pairs.AIR-Bench/qa_finance_en (AIR-Bench_24.04) — 1,585 financial texts,
used as self-supervised pairs.Objective — MultipleNegativesRankingLoss: for each (query, positive) pair
in a batch, the positive is treated as the correct answer while all other
documents in the batch act as in-batch negatives. This is the standard
contrastive objective for dense retrieval.
Hyperparameters
| Parameter | Value |
|---|---|
| Optimizer | AdamW (weight decay 0.01) |
| Learning rate | 2e-4 |
| Batch size | 32 |
| Scheduler | cosine with 10% warmup |
| LoRA rank / α | 16 / 32 |
| LoRA dropout | 0.05 |
| Epochs | 15 (elbow-selected ~4–5) |
| Max grad norm | 1.0 |
| Seeds | 42, 123, 456 |
@misc{vivekkopthsd-minilm-l6-v2-taxfinance-lora,
title = {MiniLM-L6-v2 LoRA: Tax and Finance Retrieval},
author = {Vivek},
year = {2026},
url = {https://huggingface.co/vivekkopthsd/minilm-l6-v2-taxfinance-lora}
}