sec-edgar-gpt-124m-hf

SEC-EDGAR GPT-2 124M

A GPT-2 (124M) model trained from scratch on SEC-EDGAR filings (10-K, 10-Q, 8-K, etc.) using nanoGPT.

Model Details

Training

• Framework: nanoGPT (Karpathy's)
• Dataset: SEC-EDGAR filings (financial disclosures, annual/quarterly reports)
• Tokenizer: GPT-2 BPE (tiktoken)

Usage

from transformers import GPT2LMHeadModel, GPT2Tokenizer

model = GPT2LMHeadModel.from_pretrained("lzwjava/sec-edgar-gpt-124m-hf")
tokenizer = GPT2Tokenizer.from_pretrained("lzwjava/sec-edgar-gpt-124m-hf")

prompt = "The company reported total revenue of"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.8, top_k=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Use

This model is trained for research and educational purposes — demonstrating nanoGPT training on domain-specific financial text. It is not suitable for production financial analysis or advice.

Limitations

• Trained on a subset of SEC filings; may not generalize to all financial domains
• No RLHF or instruction tuning — raw language model only
• Generated text may contain factual inaccuracies