finetuned-deepseek-finance

📄 Model Card for sharybhatti/finetuned-deepseek-finance

🔍 Model Details

Model Description

Developed by: Shaharyar Bhatti

Shared by: Groot AI (https://github.com/sharrybhatti)

Model type: Causal Language Model (LLM) for Financial Text Generation

Language(s): English

License: Apache 2.0

Finetuned from: deepseek-ai/deepseek-llm-7b using PEFT (LoRA adapters)

🛠️ How to Get Started

from transformers import pipeline

pipe = pipeline( "text-generation", model="sharybhatti/finetuned-deepseek-finance", trust_remote_code=True )

prompt = "Your Finance Related Query" output = pipe(prompt, max_new_tokens=100) print(output[0]["generated_text"])

🚀 Uses

Direct Use

Financial report and balance sheet summarization

Corporate financial Q&A assistant

Financial forecasting support in analytical tools

Downstream Use (optional)

Fine-tune further on domain-specific finance datasets

Integrate into financial analytics platforms and banking decision systems

Out-of-Scope Use

General-purpose text generation unrelated to finance

Use for generating or suggesting investment decisions without human oversight

⚠️ Bias, Risks, and Limitations

Model may reflect biases in the financial data it was trained on.

Should not be solely relied upon for critical financial decisions.

May hallucinate or generate inaccurate financial interpretations if misused.

Recommendations

Always validate outputs with human financial experts.

Use in conjunction with structured data and visual analytics when available.

🛠️ How to Get Started

from transformers import pipeline

pipe = pipeline( "text-generation", model="sharybhatti/finetuned-deepseek-finance", trust_remote_code=True )

prompt = "Summarize the balance sheet for Q4 2023:" output = pipe(prompt, max_new_tokens=200) print(output[0]["generated_text"])

📚 Training Details

Training Data

A subset of corporate balance sheets and financial Q&A datasets

Cleaned, tokenized, and formatted for instruction tuning

Preprocessing

Text normalization

Formatting in ChatML / Alpaca-style prompt-response format

Training Hyperparameters

PEFT version: 0.15.2

Batch size: 4

Epochs: 3

LR: 2e-4

LoRA rank: 8

Base model: deepseek-ai/deepseek-llm-7b

⚙️ Technical Specifications

Model Architecture and Objective

Transformer-based decoder-only architecture

Fine-tuned on financial data using parameter-efficient fine-tuning (LoRA)

Objective: Instruction-following and financial text generation

Compute Infrastructure

Hardware: NVIDIA A100 (40 GB) via Google Colab Pro+

Software: Transformers, PEFT, Accelerate, PyTorch 2.0

📈 Evaluation

Testing Data

Held-out set of balance sheet Q&A pairs and financial documents

Factors Evaluated

Accuracy of financial terminology

Relevance of generated summaries

Faithfulness to factual information

Metrics

BLEU: 0.0576

ROUGE-1: 0.2303

ROUGE-2: 0.1226

ROUGE-L: 0.1938

BERTScore (F1): 0.8559

🌍 Environmental Impact

Estimated using the ML CO2 Impact Calculator (Lacoste et al., 2019):

Hardware: A100 GPU

Hours used: ~8 GPU hours

Cloud Provider: Google Cloud

Compute Region: US-central

Carbon Emitted: ~4.8 kg CO₂

📚 Citation

@misc{sharybhatti2025finetuneddeepseekfinance, title={Finetuned DeepSeek Finance}, author={Shaharyar Bhatti}, year={2025}, howpublished={\url{https://huggingface.co/sharybhatti/finetuned-deepseek-finance}}, }

👨‍💻 Model Card Author

Shaharyar Bhatti (Groot AI)

Contact: [email protected]

• PEFT 0.15.2