minicpm5-gguf-stock-analyst

MiniCPM5-1B Financial Trading R1 Reasoning — Reasoned Stock Analyst (GGUF + LoRA), Independently Audited

Drop-in, ready-to-run GGUF (Q4_K_M / Q8_0) + LoRA adapter for MiniCPM5-1B, fine-tuned on 50K+ financial samples via SFT + DAPO reinforcement learning for stock direction signals — quantized and ready for llama.cpp out of the box. This card also comes independently audited: every benchmark number below has been re-verified on proper held-out data, not just taken at face value, so you know exactly what you're deploying instead of trusting an unverified claim.

⚡ Want faster local inference? A companion repo, minicpm5-MTP-GGUF-stock-analyst, bundles this same model with trained Medusa/MTP-style speculative decoding heads — up to 85%+ draft match rate on stock-domain prompts, meaning most forward passes become skippable with zero change to output quality. Same weights, same license, meaningfully faster generation.

Why this repo is worth your download

• Real, working quantized assets — Q4_K_M and Q8_0 GGUF ready for llama.cpp, plus the LoRA adapter for anyone who wants to continue fine-tuning on top of this checkpoint rather than starting from the base model cold.
• A documented, independently re-verified case study in catching eval leakage — why the training-set number was wrong, and the exact re-evaluation methodology used to catch it. Useful reference if you're building your own eval pipeline and want to avoid the same mistake (training-data-vs-held-out-set confusion).
• A full negative-results log (What We Tried and Failed, below) covering six different RL/fine-tuning approaches that didn't beat the baseline — saves you from re-running the same experiments.

Honest evaluation note

The originally reported accuracy was measured on training data, not a proper held-out set. Independent re-evaluation on val.jsonl shows ~33% held-out accuracy — essentially matching the base model (30%). The model memorized training examples but did not learn to generalize. We're publishing these corrected numbers for transparency; see Re-evaluation Results below for the full methodology.

Re-evaluation Results (Corrected)

Re-evaluated by downloading the adapter from HuggingFace, loading it via unsloth's native method, and running inference on val.jsonl (3,504 samples, never seen during training).

Per-class breakdown (200 samples, simple prompt)

The model has a strong BUY bias — it over-predicts BUY and under-predicts SELL/HOLD, similar to the base model's behavior.

What that number actually measured

The v1 eval used samples[-200:] — the last 200 items from the shuffled training data. These samples were seen during SFT and/or DAPO training. The model memorized these answers but didn't learn generalizable stock prediction patterns.

FAQ

Should I use this to actually predict stocks? No — held-out accuracy (~33%) is essentially random for a 3-class task. This is published as a transparent research artifact and eval-methodology case study, not a trading tool.

What's the difference between this and the MTP/GGUF-speedup repo? Same model, same weights. The MTP companion repo adds trained speculative-decoding heads on top for faster local generation — it doesn't change or fix the accuracy story, it's a separate, orthogonal inference-speed improvement.

What did you try to fix the accuracy, and did any of it work? Six different approaches (entropy minimization, curriculum sampling, iterative DPO, offline rejection sampling, higher LoRA rank, pairwise rewards) — see What We Tried and Failed below. None beat the baseline; all six are documented so you don't have to re-run them.

Versions

*v2 not yet re-evaluated on val.jsonl but expected to be similar since it's built on v1.

What Went Wrong

1. Eval on training data: The headline number was never validated on held-out data. The model learned to repeat training answers, not reason about stocks.
2. Dataset signal quality: The training data maps sentiment labels (positive/negative/neutral from news) to trading signals (BUY/SELL/HOLD). This mapping is lossy — positive news doesn't always mean BUY (stock may already be priced in).
3. 1B model capacity: A 1B parameter model with LoRA may not have enough capacity to learn the complex, noisy relationship between financial indicators and future price direction.
4. SFT + DAPO teaches format, not reasoning: SFT taught the model to output BUY/SELL/HOLD. DAPO reinforced correct answers on training data. Neither taught the model to actually reason about financials.

What We Tried and Failed (v5–v7)

All experiments were trying to beat the (invalid) training-set number. None succeeded even on training data:

• 1-Shot Entropy Minimization (+25pts on math) — collapsed to ALL HOLD on stocks
• Curriculum Sampling — combined with shorter DAPO, couldn't recover from entropy collapse
• Pairwise length reward + step verification — class collapse to SELL/HOLD (36%)
• Iterative DPO — model too weak after SFT to generate preference pairs (0/300 correct)
• Offline rejection sampling — 26% accuracy, model too weak to generate quality winners
• LoRA r=128 (4x capacity) — 21% accuracy with insufficient training steps

Key lesson: Research papers showing huge gains on math/reasoning don't transfer to noisy 3-class stock classification. At 1B scale, the model doesn't have enough latent financial knowledge for RL to unlock.

Training

Architecture

• Base model: openbmb/MiniCPM5-1B (1.08B params, Llama architecture)
• Method: LoRA r=32, alpha=64, vanilla LoRA (no DoRA)
• Target modules: q/k/v/o/gate/up/down projections
• Trainable params: ~1.2% of total

Training Pipeline

Phase 1: SFT Warmup (200 steps, ~3 min)
  - Teaches BUY/SELL/HOLD output format
  - 70% financial + 30% general data (anti-forgetting)
  - Loss: 3.25 -> 1.65

Phase 2: DAPO Reinforcement Learning (1500 steps, ~1.5 hr)
  - 3000 balanced samples (1000 per class)
  - Multi-level reward function with variance shaping
  - Mean reward: 0.3 -> 0.8 during training

Phase 3: Emergent Reasoning — v2 only (375 steps, ~45 min)
  - max_completion_length: 256 (vs 32 in Phase 2)
  - Based on DeepSeek-R1-Zero / Microsoft RLVR findings

Dataset (50K+ samples from 5 sources)

Hardware

• Kaggle T4 GPU (16GB)
• Total training time: ~2.5 hours (all 3 phases)

Usage

With llama.cpp

./llama-cli -m MiniCPM5-1B.Q4_K_M.gguf \
  -p "<|system|>You are a financial analyst. Predict: BUY, SELL, or HOLD.<|end|><|user|>NVDA $148. RSI 72, MACD bullish, PE 30, ROE 115%, 122% growth. Beat 4/4. 90% buy. AI chip launch. BUY/SELL/HOLD?<|end|><|assistant|>" \
  -n 32

With Python (transformers + PEFT)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-1B")
model = PeftModel.from_pretrained(base, "ewinregirgojr/minicpm5-gguf-stock-analyst/adapter")
tokenizer = AutoTokenizer.from_pretrained("ewinregirgojr/minicpm5-gguf-stock-analyst/adapter")

Faster inference (speculative decoding)

For meaningfully faster local generation on stock-analysis prompts with zero change to output, see the MTP/Medusa speedup companion repo — same weights, additional trained draft heads.

Files

v1 — Single-word predictions

v2 — With reasoning

Research Behind This

• DAPO (ByteDance, 2025) — 50% fewer steps than GRPO
• DeepSeek-R1-Zero — Emergent reasoning from pure RL
• Trading-R1 (UCLA/Stanford, 2025) — Financial trading with GRPO on 4B model

Related

• minicpm5-MTP-GGUF-stock-analyst — same model + speculative decoding speedup heads
• LFM2.5-Stock-Breakout-V13 — separate stock-signal model, different base architecture, same commitment to honest re-evaluation

Limitations

• ~33% held-out accuracy — essentially random for 3-class classification. Do not use as a trading signal.
• The model memorized training data but did not generalize to unseen financial data.
• Trained primarily on US equities sentiment data.
• The model outputs a direction signal, not price targets or position sizing.
• Past performance does not guarantee future results.

License

Apache 2.0 (same as base model)