Phi3-Mini-FinSight-FinancialQA

Phi-3-Mini FinSight Financial Q&A Assistant

This model is a fine-tuned version of microsoft/Phi-3-mini-4k-instruct specialized for financial question answering. It serves as the core reasoning engine for the FinSight 360 financial intelligence system.

Model Details

Model Description

A 3.8B parameter language model fine-tuned using LoRA (Low-Rank Adaptation) on financial Q&A data. The model is optimized to answer questions about investments, banking, personal finance, and corporate finance topics.

• Developed by: sweatSmile
• Model type: Causal Language Model (Decoder-only Transformer)
• Language(s): English
• License: MIT
• Finetuned from model: microsoft/Phi-3-mini-4k-instruct

Model Sources

• Repository: [More Information Needed]
• Base Model: microsoft/Phi-3-mini-4k-instruct

Uses

Direct Use

This model can be used directly for:

• Answering financial and investment questions
• Explaining financial concepts and terminology
• Providing guidance on personal finance topics
• Educational purposes for financial literacy

Downstream Use

The model is designed as a component of the FinSight 360 system, which includes:

• Real-time financial data retrieval (RAG architecture)
• Risk assessment and sentiment analysis
• Entity extraction from financial documents
• Interactive financial dashboard

Out-of-Scope Use

• Not financial advice: This model is for educational and informational purposes only
• Not for trading decisions: Should not be used as sole basis for investment decisions
• Not licensed advice: Does not replace consultation with qualified financial advisors
• Not for emergency financial situations: Cannot provide real-time crisis management

Bias, Risks, and Limitations

• Trained on only 500 samples - limited coverage of specialized financial topics
• May not reflect the most current market conditions or regulations
• Potential bias toward certain financial instruments or strategies present in training data
• Cannot access real-time market data or perform live calculations
• May generate plausible-sounding but incorrect information (hallucinations)

Recommendations

Users should:

• Verify all financial information with qualified professionals
• Not use for actual investment or financial decisions without expert consultation
• Be aware of the model's training data limitations
• Cross-reference answers with authoritative financial sources

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "sweatSmile/Phi3-Mini-FinSight-FinancialQA"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)

prompt = """<|system|>
You are FinSight, an expert financial advisor.<|end|>
<|user|>
What's the difference between a Roth IRA and a Traditional IRA?<|end|>
<|assistant|>
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

Training Data

Fine-tuned on 500 samples from FinGPT/fingpt-fiqa_qa, which contains financial questions and expert answers covering:

• Investment strategies
• Banking and credit
• Personal finance management
• Corporate finance concepts
• Market analysis

Training Procedure

Training Hyperparameters

• Training regime: bf16 mixed precision
• Epochs: 3
• Learning rate: 2e-5
• Batch size: 4 per device
• Max sequence length: 1024 tokens
• Optimizer: AdamW with cosine learning rate schedule
• Warmup ratio: 0.1
• Weight decay: 0.01
• Gradient clipping: 1.0

LoRA Configuration

• Rank (r): 8
• Alpha: 16
• Dropout: 0.1
• Target modules: All linear layers
• Quantization: 4-bit (nf4)

Speeds, Sizes, Times

• Training time: ~30 minutes on single T4 GPU
• Model size (merged): ~7GB
• Hardware: Google Colab T4 GPU (16GB VRAM)

Evaluation

Due to the small training set (500 samples), formal evaluation metrics were not computed. The model is intended as a proof-of-concept and component for the larger FinSight 360 system.

Example Outputs

Question: "How do interest rates affect stock prices?"
Response: [Model provides explanation of inverse relationship between rates and equity valuations]

Question: "What is diversification in investing?"
Response: [Model explains portfolio risk management through asset allocation]

Technical Specifications

Model Architecture and Objective

• Architecture: Phi-3 (Dense transformer decoder)
• Parameters: 3.8 billion (base model)
• Trainable parameters: ~4.2 million via LoRA (0.11% of base)
• Context window: 4,096 tokens
• Objective: Causal language modeling with cross-entropy loss

Compute Infrastructure

Hardware

• GPU: NVIDIA T4 (16GB VRAM)
• Platform: Google Colab

Software

• Transformers: 4.x
• TRL (Transformer Reinforcement Learning)
• PEFT (Parameter-Efficient Fine-Tuning)
• PyTorch: 2.x
• bitsandbytes (4-bit quantization)

Citation

BibTeX:

@misc{phi3-finsight-2025,
  author = {sweatSmile},
  title = {Phi-3-Mini FinSight Financial Q&A Assistant},
  year = {2025},
  publisher = {HuggingFace},
  journal = {HuggingFace Model Hub},
  howpublished = {\url{https://huggingface.co/sweatSmile/Phi3-Mini-FinSight-FinancialQA}}
}

Model Card Authors

sweatSmile

Model Card Contact

Available via HuggingFace profile