LFM2.5-2.6B-Finance

LFM2.5-2.6B-Finance

How this model was built — an autonomous-agent experiment

This fine-tune was performed autonomously by an AI agent — DeepSeek V4 Flash 0731 (High), running as Hermes Agent on an Ubuntu VM with GPU passthrough (an NVIDIA GTX 1080) — and is intended to test whether an agent can improve another AI model without human intervention.

The entire run was driven by a single prompt. The agent independently selected the base model and training strategy (QLoRA on the chat variant of LFM2.5-2.6B), discovered and downloaded suitable public datasets, chose the hyperparameters, trained the model overnight, and benchmarked it end-to-end — with no human intervention along the way.

The original prompt was:

Create a small LLM that excels in financial reasoning. The model will keep as much performance as possible in general reasoning, knowledge and understanding, but will have its financial knowledge improved. This will include general knowledge about how financial markets work, reasoning on how to use financial markets, how to succeed as a trader, and how it all works. The model should be capable of generating an answer to a complex financial question without issue, and even be able to reason about financial data professionally and determine actions on markets (stocks, bonds, etc) in a way that it can safely and/or aggressively make assumptions about future results or possibilities. The only hard rules I have set are: Do not generate synthetic data, either locally or with openrouter, keep monitoring progress and report back to me, and set safe limits on resource usage to prevent the machine from crashing. Use the base model LiquidAI/LFM2.5-2.6B-Base OR alternatively the non-base version if you want to start from a model that already has exceptional agentic capabilities.

The prompt also imposed hard rules on the process, all of which were honored:

• No synthetic data — only public, primary-source datasets were used; nothing was generated locally or via any LLM API.
• Monitor progress and report — the run was tracked live and reported on throughout.
• Safe resource limits — hard caps were enforced so training could never crash the host machine.

The prompt offered the choice between the base model and its non-base (chat) variant; the agent chose the non-base chat version in order to start from a model that already has instruction-following and agentic capabilities.

---

A 2.6B-parameter decoder-only language model for financial reasoning and market analysis, created by parameter-efficient fine-tuning (QLoRA) of the LiquidAI/LFM2.5-2.6B chat model on a mix of public, primary-source financial reasoning datasets.

The goal of the fine-tune was to strengthen financial knowledge and reasoning (how markets work, how to reason about them, valuation, and professional analysis of financial data) while preserving the base model's general reasoning, knowledge, and conversational ability.

This is a chat model (not a base/pre-trained-only model): it uses the standard instruction/chat template and expects user/assistant turns.

---

Model Details

License note

The base model is released under the LFM Open License v1.0 (Liquid AI, Inc.). This checkpoint is a derivative work fine-tuned from it and is distributed under the same terms — review the base model's license before commercial use or redistribution. Several constituent datasets are non-commercial (see the Data section); research and non-commercial use is recommended.

---

Intended Use

• Answering questions about financial markets, instruments, and corporate finance (valuation, earnings quality, fixed income, options, market microstructure, etc.)
• Reasoning about financial data — e.g. numeric questions grounded in earnings-call tables and financial statements
• Professional-style analysis and scenario reasoning over stocks, bonds, and macro topics

Out of scope / limitations

• Not financial advice. This model produces text, not a fiduciary recommendation. Outputs can be incorrect, incomplete, or outdated. Always verify against primary sources and licensed financial professionals.
• At 2.6B parameters it is a small model; it does not match frontier models on open-ended reasoning, and its numeric exact-match performance over tables is modest (see Evaluation).
• The fine-tune retains the base chat model's safety alignment, including its refusal to provide personalized/allocation-specific investment advice; it will discuss markets and reasoning but may decline direct "advise me" requests.
• Truthfulness calibration (TruthfulQA) declined slightly relative to the base model (see Evaluation) — treat outputs with appropriate scrutiny.
• Trained primarily on English data.

---

Training Procedure

Method

Parameter-efficient fine-tuning with QLoRA:

• Base weights quantized to 4-bit NF4 with double quantization
• LoRA adapters on attention projections (q/k/v/o_proj), the conv-block projections (in_proj/out_proj), and all MLP gates (w1/w3/w2)
• Adapters trained, then merged back into full fp16 weights for export
• Mixed precision: fp16 compute (the training hardware has no bf16 support)

Hyperparameters

Training objective: causal language modeling over chat-template-formatted user/assistant message sequences (standard SFT-style full-sequence loss).

Compute

• Hardware: a single NVIDIA GTX 1080 (8 GB VRAM), Pascal (sm_61), fp16 compute
• Wall-clock time: ~3.8 hours for one epoch
• Peak memory during training: ~7.7 GB VRAM / ~3.4 GB system RAM

---

Training Data

Training set: 10,496 examples, all from public, primary-source datasets — no synthetic or LLM-generated data was used.

Evaluation uses held-out, non-overlapping splits: FinQA test (1,147), ConvFinQA test (1,490), and a held-out phrasebank subset (500).

Redistribution note: two datasets are CC BY-NC-SA and ConvFinQA is released for research. If you intend commercial use, verify the underlying dataset licenses and, where required, fine-tune on appropriately licensed data.

---

Evaluation

Generation-based evaluation with an identical protocol for base vs. fine-tuned model (greedy decoding; numeric exact-match on FinQA/ConvFinQA; sentiment accuracy on phrasebank). General reasoning measured with lm-evaluation-harness (0-shot).

Financial benchmarks

General reasoning & knowledge (0-shot)

The fine-tune materially improved structured financial reasoning and cleanly formatted numeric answers, and it preserved or improved most general-reasoning batteries, at the cost of a reduction in truthfulness calibration (TruthfulQA).

---

Usage

This is a standard transformers causal-language-model checkpoint. Load it the same way you would load any Hugging Face text-generation model, then apply the model's built-in chat template (which supports user/assistant turns) with your tokenizer. For example, feed it a prompt rendered through apply_chat_template with add_generation_prompt=True; the model generates the assistant reply.

Because it is a small model, it also runs well with the standard local-inference tooling (transformers, and community GGUF-quantization toolchains) — suitable for running on modest hardware or as a lightweight local analyst assistant.

Recommended generation settings (inherited from the base chat config): temperature ~0.3, top-k 40, repetition penalty ~1.05 for open-ended analysis; greedy decoding for strict numeric extraction tasks.

---

Authors & Contact

Please open an issue in this repository for questions, bug reports, or collaboration.

---

Acknowledgments

• Liquid AI for the base LFM2.5-2.6B model (LFM Open License v1.0)
• The authors of FinQA, ConvFinQA, financial_phrasebank, and databricks-dolly-15k for releasing their datasets
• The QLoRA authors and the PEFT / transformers / bitsandbytes / lm-evaluation-harness communities for the open tooling used to produce this model

---

Disclaimer

This model is provided "as is" without warranty of any kind. It is an experimental research artifact and is not a substitute for professional financial advice, a licensed advisor, or a trading system. Do not make financial decisions based solely on its output. The authors are not liable for any loss arising from use of this model.