SorosAdvisor

SorosAdvisor: Financial Advisory Model Inspired by George Soros

A fine-tuned FLAN-T5-Base model that emulates George Soros's investment philosophy, providing financial insights and advisory responses in his distinctive analytical style.

Model Details

Model Description

SorosAdvisor is a sequence-to-sequence model fine-tuned on a curated dataset of question-answer pairs that capture George Soros's investment principles, including his famous theory of reflexivity, risk management strategies, and psychological approach to trading. The model generates detailed, contextual responses to financial and investment-related questions.

• Model type: Encoder-Decoder (Seq2Seq)
• Language(s): English
• License: MIT
• Finetuned from: google/flan-t5-base

Model Sources

• Repository: GitHub - SorosAdvisor
• Base Model: google/flan-t5-base

Uses

Direct Use

The model can be used directly for:

• Generating investment philosophy insights in the style of George Soros (For educational purposes only)
• Educational purposes to understand Soros's trading psychology and principles
• Exploring concepts like reflexivity, risk management, and market psychology (For educational purposes only)
• Research on financial NLP and domain-specific fine-tuning

Downstream Use

• Integration into financial education platforms (Refer Out-of-Scope use below)
• Chatbot backends for investment philosophy discussions (Refer Out-of-Scope use below)
• Research tools for studying investment strategies

Out-of-Scope Use

⚠️ This model should NOT be used for:

• Actual financial advice or investment decisions
• Trading recommendations or portfolio management
• Any use case where financial loss could occur
• Replacing professional financial advisors
• Making real-world investment choices

This is an educational and research model only.

Bias, Risks, and Limitations

Known Limitations

1. Not Real Financial Advice: The model generates text based on training data and does not have access to real-time market information or personalized financial situations.

2. Single Philosophy Bias: The model is trained exclusively on George Soros's investment philosophy and may not represent diverse or opposing investment strategies.

3. Temporal Limitations: Training data reflects historical perspectives and may not account for current market conditions or regulations.

4. Hallucination Risk: Like all language models, it may generate plausible-sounding but factually incorrect information.

5. Limited Context: Max input length of 512 tokens may truncate complex questions.

Risks

• Users may incorrectly interpret outputs as actionable financial advice
• The model reflects one investor's philosophy which may not suit all investment goals
• Generated content should be verified against authoritative sources

Recommendations

• Always consult qualified financial professionals for investment decisions
• Use this model for educational and research purposes only
• Cross-reference any information with reliable financial sources
• Do not use outputs for actual trading or investment activities

How to Get Started with the Model

Installation

1. Clone the repository

git clone https://huggingface.co/coffeewithyogurt/SorosAdvisor

1. Create and activate the virtual environment (Install uv from here if not already installed)

uv venv finv3_env --python 3.12

# For Linux/macOS
source finv3_env/bin/activate

# For Windows
finv3_env\Scripts\activate

1. Install the dependencies

uv sync --active

# Alternatively, you can also install via pip
uv pip install -r requirements.txt

1. Set the API keys

cp .env.example .env

1. Run the training script

python train.py

1. Run the inference script

python inference.py

---

Basic Usage (Inference)

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch

# Load model and tokenizer
model_path = "path/to/sorosT5Base_Finetuned/model"
model = AutoModelForSeq2SeqLM.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)

# Set device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
model.eval()

# Prepare input
prompt = "You are a financial advisor embodying George Soros's investment philosophy. Answer this question with detailed insights:"
question = "How does Soros view risk management?"
input_text = f"{prompt} {question}"

# Tokenize
inputs = tokenizer(
    input_text,
    max_length=512,
    truncation=True,
    return_tensors="pt"
).to(device)

# Generate response
with torch.no_grad():
    outputs = model.generate(
        input_ids=inputs['input_ids'],
        attention_mask=inputs['attention_mask'],
        max_length=256,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        top_k=50,
        repetition_penalty=1.2,
        no_repeat_ngram_size=3,
        min_length=30
    )

# Decode response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Generation Modes

The model supports two generation modes:

1. Sampling Mode (Creative/Diverse)

generation_kwargs = {
    "max_length": 256,
    "do_sample": True,
    "temperature": 0.7,
    "top_p": 0.9,
    "top_k": 50,
    "repetition_penalty": 1.2,
    "no_repeat_ngram_size": 3,
    "min_length": 30
}

2. Beam Search Mode (Deterministic/Consistent)

generation_kwargs = {
    "max_length": 256,
    "do_sample": False,
    "num_beams": 4,
    "length_penalty": 1.0,
    "repetition_penalty": 1.2,
    "no_repeat_ngram_size": 3,
    "min_length": 30,
    "early_stopping": True
}

Training Details

Training Data

The model was fine-tuned on a custom dataset (QuestionsParaphrased.csv) containing ~600 question-answer pairs covering George Soros's investment philosophy across multiple categories:

Data Format:

• Input: Financial/investment questions with system prompt
• Target: Detailed responses in Soros's analytical style

Training Procedure

Preprocessing

• Questions and answers cleaned for empty/null values
• System prompt prepended to all inputs:

  "You are a financial advisor embodying George Soros's investment philosophy. Answer this question with detailed insights:"

• Dynamic padding using DataCollatorForSeq2Seq

Training Hyperparameters

Evaluation

Metrics

The model is evaluated using ROUGE scores, which measure the overlap between generated and reference texts:

Results

Metrics derived from the final evaluation step of the Full Fine-Tuned run

Technical Specifications

Model Architecture and Objective

• Architecture: T5 Encoder-Decoder Transformer
• Parameters: ~248M (FLAN-T5-Base)
• Objective: Conditional text generation (Seq2Seq)
• Vocabulary Size: 32,128 tokens (SentencePiece)

Compute Infrastructure

Hardware

• GPU: NVIDIA GPU with CUDA support (recommended)

Software

• Framework: PyTorch 2.7+
• Library: Hugging Face Transformers 4.40+
• Python: 3.12+
• Package management: uv

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator.

• Hardware Type: NVIDIA GPU
• Hours used: Varies by hardware (~1-2 hours typical)
• Cloud Provider: Local / On-premise
• Compute Region: Varies
• Carbon Emitted: Estimated based on hardware and duration

More Information

Project Structure

• This repo contains 2 implementations of fine-tuning (Full-FineTuned, LoRA-FineTuned), and 1 implementation of Vanilla Transformer (implementation of Attention-is-all-you-need paper)

SorosAdvisor/
├── Full-FineTuned/
│   └── (remaining files)
├── LoRA-FineTuned/
│   └── (remaining files)
├── Vanilla-Transformer
    └── (remaining files)

Example Questions

The model can answer questions like:

• "How does Soros apply self-awareness to his trading decisions?"
• "What is the theory of reflexivity and how does it apply to markets?"
• "How does Soros manage risk in volatile markets?"
• "What role does psychology play in Soros's investment approach?"

Citation

BibTeX:

@misc{sorosadvisor2025,
  author = {CoffeeWithYogurt},
  title = {SorosAdvisor: A Fine-tuned FLAN-T5 Model for George Soros Investment Philosophy},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/coffeewithyogurt/soros-advisor}}
}
@article{2020t5,
  author  = {Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu},
  title   = {Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer},
  journal = {Journal of Machine Learning Research},
  year    = {2020},
  volume  = {21},
  number  = {140},
  pages   = {1-67},
  url     = {http://jmlr.org/papers/v21/20-074.html}
}

Contact

• Please contact by opening an issue on the GitHub repository

---

Disclaimer: This model is for educational and research purposes only. It does not provide real financial advice. Use at your own risk. Always consult qualified financial professionals for investment decisions.