Downloads · 30 days
18
9% of all-time downloads
ayoubkirouane/Sophea-Nemo-3.5-Lightning-v1
Sophea-Nemo-3.5-Lightning-v1 is a text generation model from ayoubkirouane. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<div style="display:flex;align-items:center;gap:18px;padding:16px 20px;margin-bottom:16px;border-radius:14px;background:linear-gradient(100deg,e8f5ec 0%,f7fbf8 62%);border:1px solid c6e7d1"<img alt="KIEFERSA" style="f…
Downloads · 30 days
18
9% of all-time downloads
All-time downloads
200
Public
Parameters
31.6B
65.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors65.8 GB · 100%
How the weights are stored.
BF1632.9B · 100%
From the Hugging Face model README
📄 Released with the paper Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See (arXiv:2608.17744). Every number on this card is measured, contextualized, and caveated in the paper; read it for the full evaluation protocol.
<think> blocks and follows the question's language; direct mode is clean (zero unrequested traces across the study's <think>-carrier direct-mode generations, n=9,751)Serve thinking. Leave thinking enabled so the trace is produced; it follows the question's language. Family environment note: the study's NemotronH sibling required
trust_remote_code=Falseat generation time under transformers 5.12.1; verify generation on your stack before deploying.
It does not obey an instruction to reason in English on a Greek question (0.0% override; the reverse direction works at 92.7%). Do not deploy it where an operator must force English traces. Not evaluated for safety-critical, legal, or medical use.
Fine-tuned from Nemotron-3.5-Lightning-30B-A3B, released as a third matched checkpoint of the language-matched recipe, not a full replication of the Qwen and Gpt-OSS studies.
Arrows mark the desirable direction for every metric: ↑ higher is better, ↓ lower is better.
Fidelity measures 98.06% on the identical probe (printed as 98.1 in the paper's family table), with zero in-question switches and 100% English-control compliance.
Logic is a capability floor shared with its own base (23.8 vs 25.0 macro-recall; logic is the hardest axis for every arm in the study), not a fine-tuning regression: the release's logic fallback is a mild 5.3%, the lowest of the five releases. Its fallback cost sits instead on math and commonsense (16.1 / 14.7%). Like its Nano sibling, this family returns 5–9% of override-probe rows too short to score.
This suite exists to answer one question: did the Greek fine-tune damage general ability? It did not. Against its own base, across 14 bilingual NLU benchmarks: +1.7 Greek / −1.1 English, both within the range seed noise alone produces on this class of suite. Read the numbers below as a forgetting check, not a capability leaderboard.
<div style="overflow-x:auto"> <table style="width:100%;border-collapse:collapse;font-size:13px;font-variant-numeric:tabular-nums;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif"> <thead><tr> <th style="padding:10px 8px;text-align:left;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;">Benchmark (Titan-1 suite)</th> <th style="padding:10px 8px;text-align:center;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;">Base</th> <th style="padding:10px 8px;text-align:center;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;">Sophea-Nemo-3.5-Lightning-v1</th> </tr></thead><tbody> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">ARC-Challenge-el</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">68.7</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">72.8</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">ARC-Easy-el</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">75.8</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">81.4</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Belebele-el</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">81.3</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">83.3</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">GreekMMLU</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">64.2</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">62.2</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">HellaSwag-el</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">44.5</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">51.2</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Medical-MCQA-el</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">41.7</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">38.2</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">TruthfulQA-el</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">34.2</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">33.7</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Winogrande-el</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">55.8</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">60.1</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">MMLU-el</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">51.3</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">49.6</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);"><em>Greek macro</em></td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);"><em>57.5</em></td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;"><em>59.2</em></td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">ARC-Challenge (en)</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">92.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">89.1</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">ARC-Easy (en)</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">97.8</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">96.4</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">HellaSwag (en)</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">72.7</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">75.9</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Winogrande (en)</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">72.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">73.3</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">MMLU (en)</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">75.8</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;">69.7</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);"><em>English macro</em></td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);"><em>82.0</em></td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;"><em>80.9</em></td></tr> </tbody></table></div>Register control improves 37 → 43 of 46 and grammaticality roughly doubles, 12 → 29 of 58 (judge-scored counts, directional evidence only). In direct (non-reasoning) mode on the same suite the release reads +1.7 against its own base.
Serve with vLLM (OpenAI-compatible):
vllm serve KIEFERSA/Sophea-Nemo-3.5-Lightning-v1 --served-model-name sophea-nemo-3.5-lightning-v1
Client (OpenAI SDK):
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
model="sophea-nemo-3.5-lightning-v1",
messages=[{"role": "user", "content": "Ένα κατάστημα πουλά στυλό προς 1,50 € το ένα. Πόσο κοστίζουν 12 στυλό;"}],
temperature=0.7,
)
print(resp.choices[0].message.content) # <think> trace in the question's language, then the answer
Steering the trace language (partial, measured per checkpoint): append "Think step by step IN ENGLISH." to a Greek question, or "Σκέψου βήμα προς βήμα ΣΤΑ ΕΛΛΗΝΙΚΑ." to an English one. On this checkpoint the English suffix is not obeyed (0.0%); the Greek suffix on an English question is obeyed on 92.7% of items.
This model ships the base's multi-token-prediction head: 270 mtp.* tensors (~2.7 GB, bf16)
in model-mtp.safetensors, the draft stack that config.json declares through
num_nextn_predict_layers: 1 (an attention block plus a MoE block). The head is the base
model's own; speculative decoding verifies every drafted token against the main model, so it
changes throughput only, never the output distribution.
Enable it with vLLM (0.27.1 or newer):
vllm serve KIEFERSA/Sophea-Nemo-3.5-Lightning-v1 --trust-remote-code \
--reasoning-parser nemotron_v3 \
--speculative_config.method mtp \
--speculative_config.num_speculative_tokens 3
Per vLLM's guidance for this family, MTP suits low-latency serving; for maximum batch throughput, run without speculative decoding.
The fine-tuning additions are released under Apache-2.0. The model inherits the base model's license terms (nvidia/Nemotron-3.5-Lightning-30B-A3B); verify the base model's license before use.
This model is a release artifact of arXiv:2608.17744. If you use it, please cite:
@misc{kirouane2026thinking,
title={Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See},
author={Ayoub Kirouane and Christos Petrocheilos},
year={2026},
eprint={2608.17744},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2608.17744},
}