fin-sent-tiny

fin-sent-tiny

A tiny, from-scratch text classifier that labels a financial-news-style sentence as positive, neutral, or negative. Built as a learning exercise: custom PretrainedConfig + PreTrainedModel architecture (embedding -> mean pooling -> small MLP), trained on ~90 hand-labeled sentences.

This is intentionally small (~21k parameters) so the whole pipeline — tokenizer training, model definition, training loop, saving, and reloading — runs in seconds on a CPU with no GPU required. It is not meant to be state-of-the-art; treat it as a working reference for how a Hugging Face-compatible model repo fits together.

Usage

This model uses a custom architecture, so AutoModel.from_pretrained alone won't know how to build it — you need the modeling.py file (included in this repo) alongside the weights:

from transformers import PreTrainedTokenizerFast
from modeling import FinSentClassifier  # from this repo

tokenizer = PreTrainedTokenizerFast.from_pretrained("YOUR_USERNAME/fin-sent-tiny")
model = FinSentClassifier.from_pretrained("YOUR_USERNAME/fin-sent-tiny")

inputs = tokenizer(["Profits soared to a record high this quarter."], return_tensors="pt")
logits = model(**inputs).logits
pred = logits.argmax(-1).item()
print(model.config.id2label[pred])

Training data

~90 hand-written sentences in the style of the Financial PhraseBank benchmark (Malo et al., 2014), evenly split across the three labels. See data.py.

Known limitations

Trained on a tiny, hand-written dataset — expect it to make mistakes on real-world financial text, especially longer or more nuanced sentences. Held-out test accuracy was ~61% against a 33% random baseline on 18 examples, which is enough to show learning happened, not enough to trust in production. Swap in more real labeled data (e.g. the full Financial PhraseBank) to improve it meaningfully.