Downloads · 30 days
24
5% of all-time downloads
durganani60/qwen2.5-0.5b-financial-adapted
qwen2.5-0.5b-financial-adapted is a text generation model from durganani60. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a domain-adapted version of Qwen2.5-0.5B specifically tailored for financial NLP tasks. It features a custom Byte-Level BPE Tokenizer integrated directly into the base architecture via Tokenizer Surgery…
Downloads · 30 days
24
5% of all-time downloads
All-time downloads
454
Public
Parameters
630M
1.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.3 GB · 99%
From the Hugging Face model README
This model is a domain-adapted version of Qwen2.5-0.5B specifically tailored for financial NLP tasks. It features a custom Byte-Level BPE Tokenizer integrated directly into the base architecture via Tokenizer Surgery and aligned using TRL SFTTrainer with PEFT/LoRA on Apple Silicon (MPS).
Standard LLM tokenizers often fragment specialized financial terminology (such as ticker symbols like $NVDA, key metrics like EBITDA/CAGR, and currency pairs like EUR/USD) into excessive subwords, leading to increased prompt inflation and potential [UNK] errors.
This model addresses these issues by:
Qwen/Qwen2.5-0.5Bimport torch
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load model and custom adapted tokenizer
model_id = "durganani60/qwen2.5-0.5b-financial-adapted"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto"
)
# Test prompt containing financial domain tokens
prompt = "Q3 2024 financial report: $NVDA recorded EBITDA growth above 14.5% with expanding FCF margin."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
'''