finrag-gpt-oss-120b-v2

FinRAG Trading Expert — gpt-oss-120b (v2)

A LoRA adapter that turns openai/gpt-oss-120b into a quantitative trading analyst: it audits backtests for bias, extracts implementable strategy specifications from research papers, reasons about market microstructure and derivatives, and designs risk management for live deployment.

The adapter is distilled from GLM-5.2 teacher responses over 500 full-text quantitative-finance arXiv papers, real trading-project material, and a curated slice of public finance reasoning data.

⚠️ Not investment advice. See Limitations & Risks. This model produces analysis and hypotheses, not trading signals. Nothing it outputs should be traded without independent validation.

---

What it is good at

The training data is written in an "elite quant strategist and backtest auditor" voice that thinks step by step, flags biases, and distinguishes in-sample from out-of-sample results.

---

Model Details

• Developed by: dustin2050
• Model type: LoRA adapter (PEFT) for a Mixture-of-Experts causal LM
• Base model: openai/gpt-oss-120b — 117 B total / ~5.1 B active parameters (trained through the 4-bit build unsloth/gpt-oss-120b-unsloth-bnb-4bit)
• Language: English
• License: Apache-2.0 (adapter). The base model's own license applies to the base weights.
• Finetuned from teacher: GLM-5.2 (z-ai/glm-5.2, via OpenRouter) — responses distilled into the student

LoRA configuration

---

Training Data — distill_v3 (14,452 examples)

Filtering of the external set: from 79,112 rows, corporate-finance/accounting/PE/insurance topics were removed (11,993), and the remainder was balanced across 9 quant themes (~690 each): microstructure & execution · derivatives & options · fixed income & rates · crypto/DeFi · risk & portfolio · strategy & alpha · liquidity & credit · macro & FX · behavioral markets. It was chosen because it shares this project's teacher (GLM-5.2), its <think> reasoning format, and has 0 % truncation.

Quality properties:

• 99 % of examples carry explicit <think> reasoning before the answer
• Median ~2,850 tokens per example; 99.9 % of content fits within the 12,288-token training window
• Mid-generation truncated answers removed; teacher regenerated with max_tokens=12000
• Deduplicated on the full question (a prefix-based key would have silently deleted nearly all full-text paper examples, since they share an identical instruction prefix)
• Composition deliberately keeps 58 % domain-specific trading distillation vs 41 % general quant knowledge so the specialised auditing behaviour stays dominant

---

Training Procedure

Why sequence length 12,288 matters

An earlier iteration of this project trained at max_seq_length=2048, which captured only ~54 % of the training content — 93 % of examples were cut mid-answer and full-text papers were truncated to ~38 % of their length. At 12,288 tokens, 99.9 % of the content reaches the model, and full papers fit whole.

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

BASE = "unsloth/gpt-oss-120b-unsloth-bnb-4bit"   # 4-bit, ~65 GB VRAM
ADAPTER = "dustin2050/finrag-gpt-oss-120b-v2"

tok = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)

messages = [{"role": "user", "content":
    "A momentum strategy on S&P 500 futures shows 340% CAGR over 4 years with 18% max drawdown "
    "and 40 trades/month. Audit it: what would you check before believing it?"}]

inputs = tok.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(inputs, max_new_tokens=2048, temperature=0.7, top_p=0.9, do_sample=True)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

With Unsloth (faster load):

from unsloth import FastLanguageModel
model, tok = FastLanguageModel.from_pretrained(
    "unsloth/gpt-oss-120b-unsloth-bnb-4bit", max_seq_length=12288, load_in_4bit=True)
from peft import PeftModel
model = PeftModel.from_pretrained(model, "dustin2050/finrag-gpt-oss-120b-v2")
FastLanguageModel.for_inference(model)

Hardware requirements

The prompt style the model was trained on: a direct analytical question, optionally with a paper excerpt or project code pasted in. It responds with <think> reasoning followed by a structured markdown answer.

---

Intended Use

In scope

• Research assistance for quantitative traders and strategy developers
• Critical review of backtests, papers, and strategy write-ups
• Educational exploration of quant finance concepts
• Generating hypotheses and implementation plans to be validated independently

Out of scope

• Generating live trading signals or portfolio allocations for real capital
• Any use as a substitute for licensed financial advice
• Fully autonomous trade execution
• Non-English use (training data is English-only)

---

Limitations & Risks

• Not investment advice. Outputs are analytical text, not recommendations. Trading involves risk of substantial loss.
• Teacher-inherited errors. The model imitates GLM-5.2; factual mistakes, invented citations, or plausible-but-wrong numerical reasoning can be reproduced confidently. Verify every number and citation.
• No verified numeric accuracy. The training data was not numerically ground-truthed; arithmetic in cost/Sharpe calculations may be wrong.
• Knowledge cut-off & drift. Papers are ~2024–2026 q-fin arXiv; market regimes and published alphas decay. The model may present decayed edges as viable.
• Single-epoch, single-run. No held-out evaluation was used during training (the gpt-oss eval path is broken in this Unsloth build), so quality is evidenced by training loss and qualitative review only. A systematic evaluation is in progress.
• Style bias. Long, structured, markdown-heavy answers with a skeptical tone — this is intended, but can read as overconfident.
• Distribution skew. 41 % of training data is general quant Q&A in a uniform "How does X impact Y when Z" template, which may bias the model toward that answer shape.

---

Acknowledgements

• Base model: OpenAI gpt-oss-120b
• Teacher: Z.ai GLM-5.2 (via OpenRouter)
• Training stack: Unsloth (Faster-MoE kernels), TRL, PEFT, Transformers
• External data: ianncity/GLM-5.2-Finance-80000x (Apache-2.0)
• Source literature: open-access q-fin preprints from arXiv and OpenAlex

Citation

@misc{finrag_gptoss120b_v2_2026,
  title  = {FinRAG Trading Expert: a GLM-5.2-distilled LoRA for gpt-oss-120b},
  author = {dustin2050},
  year   = {2026},
  note   = {LoRA adapter, distilled from 500 full-text q-fin papers and curated quant reasoning data},
  url    = {https://huggingface.co/dustin2050/finrag-gpt-oss-120b-v2}
}