KiyanNet-Finance-Base-48M

KiyanNet Finance Base 48M

KiyanNet Finance Base 48M is a compact finance-domain causal language model published in standard Hugging Face Transformers format. It is intended as a lightweight baseline for finance and India-market language modeling, continued pretraining, and downstream fine-tuning.

This is a base model, not an instruction-tuned assistant. It should not be used to provide buy/sell calls, price targets, guaranteed outcomes, or personalized investment advice.

Load With Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "techkiyan/KiyanNet-Finance-Base-48M"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

prompt = "Reliance Industries reported"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=True,
    temperature=0.8,
    top_p=0.95,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

No custom Python files are required.

Files

This public repo intentionally contains only standard loading and fine-tuning files:

• config.json
• generation_config.json
• model.safetensors
• tokenizer.json
• tokenizer_config.json
• README.md

Training scripts, logs, token shards, raw datasets, duplicate checkpoints, and custom model-code files are intentionally excluded.

Model Details

The HF parameter count is slightly higher because the public Transformers conversion uses GPT-2 modules with zero-initialized bias tensors. The converted model was checked against the original checkpoint; logits matched within normal floating-point tolerance on a fixed prompt.

Fine-Tuning Starter

from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments

repo_id = "techkiyan/KiyanNet-Finance-Base-48M"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

# Prepare a tokenized causal-LM dataset before passing it to Trainer.
training_args = TrainingArguments(
    output_dir="kiyannet-finetuned",
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
    learning_rate=5e-5,
    num_train_epochs=1,
    logging_steps=25,
    save_steps=500,
)

Benchmark Protocol

Benchmarks were run locally on CPU with torch==2.13.0.

The benchmark is intentionally finance-focused and lightweight. It is not a leaderboard result. It was designed to compare small standalone causal language models in the 30M-100M range under identical local conditions.

Model Selection

The target comparison set was 10 standalone AutoModelForCausalLM models between roughly 30M and 100M parameters.

Public finance-specific causal LMs in this exact size range are scarce. One finance-named 70M candidate, Rudra501/model70m_finance, appeared to be an adapter/training artifact rather than a standalone loadable checkpoint, so it was excluded. Larger finance models, GGUF-only models, classifiers, fill-mask models, and models above 100M parameters were also excluded.

Evaluation Methods

The benchmark uses multiple strategies:

• Category perplexity on finance-style snippets
• 20 fixed generation prompts across company, regulatory, technical-analysis, options, filing, and safety categories
• Greedy generation throughput
• Sampled generation throughput
• Repetition heuristic
• Bad-advice phrase heuristic
• Metadata-marker heuristic
• RBI role-error heuristic
• Nifty foreign-context heuristic
• Finance-tokenizer round-trip tests

10-Model Benchmark Summary

Lower perplexity is better. Higher tokens/sec is better.

Diagnostic Heuristics

The bad-advice heuristic is intentionally broad and counts appearances of words such as buy, sell, target price, stop loss, guaranteed, and risk-free. It is useful for smoke testing but should not be treated as a complete safety evaluation.

Prompt Coverage

The generation benchmark used 20 prompts:

• Reliance Industries reported
• Tata Motors quarterly results showed
• HDFC Bank announced
• Infosys management stated
• State Bank of India reported
• The Reserve Bank of India announced
• SEBI issued a circular regarding
• NSE Clearing informed members that
• Nifty 50 approached resistance while
• Bank Nifty tested support near
• RSI showed bearish divergence because
• Volume did not confirm the breakout
• IV rank differs from IV percentile because
• Rising open interest with falling futures prices indicates
• An iron condor has maximum loss when
• Put-call ratio should not be interpreted alone because
• The company announced a board meeting to consider
• The rights issue will dilute shareholders if
• The IPO risk factors indicate
• This market summary should not provide a buy or sell recommendation because

Tokenizer Round-Trip Checks

The current tokenizer preserves finance notation in all tested examples:

Sample KiyanNet Outputs

Greedy decoding examples from the benchmark:

These samples show both strengths and weaknesses. The model has finance-domain fluency and performs well on the small local perplexity benchmark, but it still invents figures, mixes jurisdictions, and can reuse generic news or filing patterns. It should be treated as a raw-mix baseline, not a final India-first analyst model.

Interpretation

KiyanNet Finance Base 48M performs best on this local 30M-100M finance-smoke benchmark by overall perplexity, company-text perplexity, regulatory-text perplexity, technical-text perplexity, and filing-text perplexity. It is also faster than DistilGPT2 while using fewer parameters.

The main weakness is options and derivatives language. DistilGPT2 and Pythia 70M Deduped v0 scored better on the options subset. This supports the next training direction: add explicit options, futures, volatility, and technical-market-state data rather than continuing only with generic financial news.

Intended Use

• Compact finance-domain language modeling experiments
• Continued pretraining
• Fine-tuning for structured finance tasks
• Lightweight local baselines
• Financial text understanding research

Not Intended For

• Buy or sell recommendations
• Entry/exit levels
• Price targets
• Guaranteed return claims
• Personalized investment advice
• Live factual claims without retrieval or external verification
• Option payoff or Greek calculation without deterministic tools

Limitations

• This is a base model, not an instruction-tuned analyst.
• It can hallucinate dates, figures, institutions, currencies, and corporate actions.
• It can mix India-specific prompts with global-news templates.
• Options and derivatives understanding is still weak.
• Generated text should be verified against reliable data sources.
• The benchmark above is a small local smoke benchmark, not a broad public leaderboard.

Recommended Next Step

Preserve this model as a 607M-token raw-mix baseline. The next stronger base model should use a balanced India-first training mix with explicit regulatory, corporate-announcement, technical-analysis, options, futures, volatility, and market-state data.