finsight-qlora-mistral

FinSight QLoRA Adapter — Mistral 7B for Financial QA

A LoRA adapter fine-tuned from Mistral-7B-Instruct-v0.3 on real question-answer pairs derived from 2023 10-K filings. Optimized for extractive financial question answering: given a passage from a 10-K and a question about it, produce a concise, grounded answer.

Part of the FinSight project.

Performance

Training Setup

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

BASE = "mistralai/Mistral-7B-Instruct-v0.3"

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, "musk1209/finsight-qlora-mistral")

context = "The Company designs, manufactures and markets smartphones..."
question = "What does the company sell?"
prompt = f"[INST] Answer the question based on the provided context from a company's 10-K filing. Be concise and factual.\n\nContext: {context}\n\nQuestion: {question} [/INST]"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Observations

Head-to-head inference tests show that the fine-tuned model produces slightly more disclosure-style, complete answers than the base Mistral 7B, but the base model already performs well on this extractive QA task. This is consistent with modern instruction-tuned LLMs being strong on extractive QA out-of-the-box; fine-tuning provides refinement rather than enabling entirely new capabilities.

Limitations

• Dataset origin: Training data is derived from 2023 10-K filings. Performance on other filing types (10-Q, 8-K, DEF 14A) or on filings outside 2023 may differ.
• Extractive bias: The model is trained to answer questions where the answer is contained in the provided context. Open-ended synthesis across multiple documents is not part of its training.
• Compute-time trade-off: Producing this adapter required ~6 hours of T4 GPU time. In practice, for many production use cases, the base Mistral 7B Instruct v0.3 with careful prompting delivers comparable quality without fine-tuning.

Citation

@misc{finsight-qlora-mistral, title = {FinSight: QLoRA-tuned Mistral 7B for Financial Question Answering}, author = {Tongaria, Muskan}, year = {2026}, url = {https://huggingface.co/musk1209/finsight-qlora-mistral} }

Honest evaluation vs base model (Path A vs Path B)

The finetuned adapter was compared head-to-head against the base Mistral 7B Instruct v0.3 with a strong system prompt (no adapter), on 6 test cases spanning basic extraction, numerical questions, risk factors, adversarial "answer not in context" prompts, exact-figure questions, and multi-fact synthesis.

Result: Path B (prompted base model) was chosen for deployment. Both approaches performed comparably on standard extraction, but the adapter demonstrated two critical failure modes:

• Hallucination on unanswerable questions. When asked about CEO compensation given only a context about products, the adapter invented a specific figure ("$39,629"). The prompted base model correctly responded that the context did not contain the information.

• Loss of reasoning on synthesis. When asked which of two segments performed better given growth rates for each, the adapter answered "Segment A" with no justification. The prompted base model gave the reasoning ("Segment A performed better, as it experienced a 12% increase compared to Segment B's 3% decrease").

Why this happened: the training dataset contains only answerable QA pairs where the answer is present in the context. The model was never trained on "should refuse" examples, so it learned to always produce an answer. This is a data-composition issue, not a QLoRA issue.

The full comparison results are in the project repository: data/path_comparison_results.json.