Downloads · 30 days
11
29% of all-time downloads
oscarz511/NanoSOTA-Qwen-0.5B-GSM8K-v1
NanoSOTA-Qwen-0.5B-GSM8K-v1 is a machine learning model from oscarz511. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a highly specialized version of Qwen/Qwen2.5-0.5B-Instruct, fine-tuned to excel at multi-step mathematical and logical reasoning. It was trained to first generate an internal monologue (<|beginofthought|...) b…
Downloads · 30 days
11
29% of all-time downloads
All-time downloads
38
Public
Parameters
494M
1000 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors988 MB · 98%
From the Hugging Face model README
This is a highly specialized version of Qwen/Qwen2.5-0.5B-Instruct, fine-tuned to excel at multi-step mathematical and logical reasoning. It was trained to first generate an internal monologue (<|begin_of_thought|>...) before providing a final, boxed answer (\boxed{...}).
This model is the result of a multi-stage training process designed to create a "NanoSOTA" (State-of-the-Art) model for its size class on reasoning tasks.
This model is intended for tasks requiring step-by-step reasoning, particularly elementary school math word problems (the GSM8K benchmark). It can be used for:
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "oscarz511/NanoSOTA-Qwen-0.5B-GSM8K-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
prompt = "A farmer has 15 sheep. All but 8 die. How many are left?"
messages = [{"role": "user", "content": prompt}]
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs, skip_special_tokens=True))
The model underwent a three-stage training process on an NVIDIA L40S GPU:
Qwen/Qwen2.5-0.5B-Instruct.HuggingFaceH4/Bespoke-Stratos-17k to learn the <|begin_of_thought|> reasoning structure.gsm8k (main) dataset to master the format and logic of word problems.