Sophea-RAG-Nemo3

KIEFERSA/Sophea-RAG-Nemo3

Sophea-RAG-Nemo3 is a text generation model from KIEFERSA. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.

<div style="display:flex;align-items:center;gap:18px;padding:16px 20px;margin-bottom:16px;border-radius:14px;background:linear-gradient(100deg,e8f5ec 0%,f7fbf8 62%);border:1px solid c6e7d1"<img alt="KIEFERSA" style="f…

Downloads · 30 days

23% of all-time downloads

All-time downloads

Public

Parameters

31.6B

63.2 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors63.2 GB · 100%

Parameter types

How the weights are stored.

BF1631.6B · 100%

Base models

nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16

Model card

From the Hugging Face model README

<div style="display:flex;align-items:center;gap:18px;padding:16px 20px;margin-bottom:16px;border-radius:14px;background:linear-gradient(100deg,#e8f5ec 0%,#f7fbf8 62%);border:1px solid #c6e7d1"><img alt="KIEFERSA" style="flex:0 0 auto;height:58px;width:58px;border-radius:12px;object-fit:contain" src="data:image/webp;base64,UklGRgQNAABXRUJQVlA4WAoAAAAQAAAAxwAAxwAAQUxQSIsKAAABsMf/nyE52q+qZjd2nluEh00OsY2zbdsOH51t27Zt2744F1vrrqrf69X1m+mumq4K/ooISG4bSZKUmIIOeag1w1Ud8wFwPDgH2Om9zYFD2AcXDIa/hdWdgAff+r6oNK7pDDzw1uPxRlQRVoccFlu3e+pj0zrksByD8huqjSFSAm0c2l+8ilrIEQxaT1qC2ljAYYJD89PnxaYQQ05spcdMp5ZEcI0f/CtqGVsygTXY4+vYJGLI4QJg7CeIxkIOFwyGvE0tLeG0Xs8rVLGFHBZb1aORCXAYcniOQdfb61FJjXaEcEb5NeuoWRI+63DhShrgrAmdtZy4iAa4kJPj0PSMOdRCjuCQO3oaDXAhR3Dg+/9Ko0/IERxgl29pDA05QgCM+ohGn7DDYPDr1EJPr2cUiT7hp5aEhvUBZMZ6AY3rC3BjkLVdga0PkFjTGUqCDykOMw9kwEVwKIL9c1zX1d/saIJUsCFvE4sntILyFYjvjgQmeFBwbCsuaAMCyldhhOqVAYERp7bm2s3MKTMoKTF6apugiEOrvaMr+XbADJq8JtXf391IQImt4aEqYpBH3kmtuaXS7E4gkRL1s73pBS8IEY1rLusYCHFi+PoQurwnkjdh+dQ2wHP+i71p/GgM/XaQggITFpzdIpaQoWL7ZlcaGtKD5LPnrBOaeB8tbO3XAzg5LylIsYe/HyY8FwuLNE4/uoSck5SkkO/38jsk2djc05uSK5uSlOvQZzsAcBESdGyLJrckP4hZkFLeGw7AuZ+kDnAXt6fVyIa0ol7u62u0SGfrri2nM5xDQpJ8oqefksbq7+hOk4Eddgmw4e7uwHLMa6TC6NEe9DLaYyfV13sY95JMv5AX4FxgK6sv7ZiFCVwkDFdIifj2EBp93GAvS6a29ivuFbRPx1Jzi135nndWsyKXMdZvxLDCo7kDlNT43Z5AjjYTFDiW6ceVFFEYNJmOqtCthdgLuCXSBLiDaIBzi5sJv5PjKRalf6PU+UNptZ0lKrZpx5WSo8wA6eLeHhBLkZiWsNGWG/P8nEmfnwySd499Og5AsMyDSye3ISuzexyKQnx7KGSfSzvS6FMEHIt+ui2wjEPrffFxUcaGg8g09IHJPqg1DgGe9Q34AeLQTSF0Wr5egOfMJlKBh5S4QQ+f9zNqqfwkS9bzMV3TCU4yHXsq1LDYut9Ti7i6G0Dzc0gHpX9kxcpvXIeqEas7QQmDNv9bhipSgYUE2PaXrUIVadJHwUhv7pWrUUU6oJAvHi0nLzUnK78fhJ7STrdWo4rCSWzNzphvFpSkvhYuGGxxv+ldDyM5DqXHzEAtdZr+HBZLr6ckShk+BAdx0O+mTCX1GSUUslcwFk8oprE9v0EtpU2/lBAAI99DLWW44AJg/EfmzKTr+0qSnb4y0SJMCMFg2FsapXTRvyY48L1/MhPCA4+t9wsKpXTVhyc4iCOmmZAUFlhsVY81oops+glTSZNTZpv1NBzw2LreVYdKard9kSyWFhMWmLgXAmg+KLtuHSpZhP5OJhi0vWC5CUn+w3Ic2l9IAhzaYXfyr1pjdttzBIfWE5bQ/OwAiwldbiMPQyZxGeBO+we1MQdYylYPNpoJGcSZ5Y6ZRs0BDhao7Z5VqKSPCA7soLz3fUc4kIFvIErpG5wD7JYX4LIB/WFw9IdGMkRfKM1ZjPxgNuZj+tOCS9xck12/NtckM6T+HSwh8r9F7z23uHpGDjTPiMwCiPjQ1TdcbzGM9XqOrgWOcSe5o/5CFRURdyNvbZYa3eO0hsxGHJIFZGQ1utxeRz+2uMettJywbCSIjG9WUssIViGpvD2wjKONFQfXAgCQdRB9AICxzIO+ALAhy7ouwNYHSB1vSsKKRSvuzsAEDx8OOgWnbr38jQFG/CC7tvaazaB8BTY+uV0RGkkg43nIsdXe0RkEVKzECOvv28JMCMbGsTU8vCUwIaB8FfkAUn1LJbAc84mbJ/33PxbD2eIoUT5rbmGe10dBH9ErOwITzBM02vYTOjKF+PpQc/sW7AchE5b9p7VLyfj7tKMZ+CFpxU3ua9GRxoVnN3fXHpzx7xsuTOPXuwFpnEyETNA4+/gmwEXoULH9sj8n5yeBhD3745CSWHzHQQvzkSXAcwCJJO7dD3uzWEKFMq24pzUl5yUVifLpjgBChAgd25KJLcnVTUWqNejdkaYgBwZSlVZc0o7U15SkrGwv9TfiK+ltzfVltCKlJW11i57sacRPUge4O7tScwpipLD+/m5mh4OAVBg90oNeTlvSnvrqGyqM+I6WCvWzfelN6Yo0O77mso4muHqNlIhvDqXRxwU2E5ZNbm1CkrdIqfGTsdTcYFcq5p3ZHLjwExXbd7vRIuoK29I9/YQmsfiHijT+diAnR+wKF/LrISIWbyh03Y7LkaMtOkn31V4mJHlC3oow70wa4IpOitXq0/EAnHsBscVTWpF1zS3u5O2hRjwg0rjq4va0YjrGoegX+5jDyv6m+vpKenjOcSgK5WNVkH3u7kZrvnPcf6q6vSWwjEPLbVFwLsNAZAJbyz6oNQ4BnvmNHyAO3RRCp+XrBXgOylZiowo8pFBvddMlc1BL7SdZsorra+sqm/5vuZFQw3IcOly2EnFNV4COl5Om2gBDvni0nLIEVSNWd4ISBpW31hjJNn+j1PlDaRWTDWt21jyTbEgfBeOCQff761FGOqib2EqOm2XyWaF+ECPbPtWISnpF8Y0f+DtqqQr1tRRoqn1Vo5TucC7de/YoPEozYHt8h1qqpP4cKgCj3sVYHOH14AJg3MeopUzbZyQEwM5foJYqk/CkUWwb/qa5zjb9UoID3+dH1FKGa8MFg74vmOfOtu9LcMgd9reZECZYbD0fbUQVpe1fS5QmJ80166gd/lrXu2tNhU3Zh5cqYzQ/b6GRoEC+eJRdvxaV1O76CZlg0OaC5SZaeEyqi9ruEtJQbNUXmVI6Xk3iXiAQDFpNWYramAUWt08nGvcCQDyj2anz6NuANaknVD3QgCryHcGh9OgZtJxaYlkkej2tUEmfEQLEAb/QUOAASxn0GqKUviI4wO7f0EBQLAqHpFEfGfERYQJc3jEXHwATwnb+2lwL3+CCweA3dGwai0tS3Nv/F1OQC+CJ9Xpe0hcYd7gRceTfRnyBxbblIzTAOcRd7Why2hwzgeDBjM531JHo4xSXnxNaTFiEKvIABmXXVNNPR25xm63aX7AccUjmaX/BSmqucZ51r6kbASLjzKMBrhi4lqoyYBkHpUbMOgCMzMg4CtEHADjLPOgLABuyrOu0fkDpmi7rAUi2WfgvYIGHvNnV3do5SH/gfEbjw1saCxGOZ6hntzWZM+xIifqNwdTChEPDD8cA8NgCDnnL/nJXaiFHxfbL/oy0f4YMNzbtqFJiIUdFCv85tTmxgEPeTxdPaEUs5OhI4coL2pJ/WAghdrbu2jLyNSqMw8Zq7+pCLeSYAPdYFbWQIyXK5/rQ6BNypER8fQi1oJLKPhoPYCzkKKnxm92AFKrgkmi/HSiIBZgEm3F0KbGQoyKN805tSizk6EjhkqktSfQJNvkB7pJ2wIyFG2JrbyynoSHkmC9hd3WjFnYweqQnDQ2h59W+1IIPAABWUDggUgIAAHAdAJ0BKsgAyAA+MRiJRCIhiBJcEAGCWlu4MAAZsUGEUQU6Vr/HVACel73mNKbZii7/0o6X9Y+s6FoAm3srqdXIcWd/45kb7mdOviNuuq60W3qXx/nI9sR42ICkrOwkSyYty0cun8cMpqXF0fh8fW3dSwaV0akfevRFNfnTLgy9bQM6IEWlj9O95hHRhKjNbDCS4qWJnQbJto47spFiqntPEoRO945gp7f+FzLmeAJQakEjD+xgQDxMOYrHw79WEkLQkKplqTo/uTX4n54eEP0N4epeSNwNE7Cl7q0Xi32YrSRma3RieLfasZELkHnrVj60WkAA/uouoblbKpSNm/wu8g0tISmLMdPrSIKVi8oMaNvyXkZc6zskreB0cAODf99ubzUwj0N8IJywGgCWpByLV1Kx+G8TMEVJa0HKGlyheflfEABBIEOGipfrRMJEDIiV7tNyTCRkHRlacBBPJNrPF6gu5KqHk0mGh2xBZYDKEWXI1A8R9WyRig0s6CIELwgJcqvHXXixg1yTYYd3Ja2rm0FNGHOb8MdNErR6nH3DYmradAb7JnP/RNqQ3gvhl/pA7ygUm6CIWAO/JF0d93ZViT6KqEgvSVdzZIwTzPR1TQ5YDMPK1u42qqN2SqTRwtzHrtold7EeIrx7plafSjlNtoSSwuC99MOigEHtk+7uPfbMdQA6uO4+wAsQ6bfVGy+dpyM63pbfBdYa2IbgqA8ZA3EUsyDx6GwlVker0XVZhE/HSCyOiMAVqYYDYXpIECKSg6k4BD3OKMJ7nJlAAAAAAA=="/><div><div style="font-size:1.24rem;font-weight:700;color:#003713;line-height:1.2">Sophea-RAG-Nemo3</div><div style="font-size:.86rem;color:#3a5a47;margin-top:2px">Greek + English grounded RAG reader — fine-tuned NVIDIA Nemotron-3 Nano 30B-A3B</div></div></div>

Sophea-RAG-Nemo3 is a Greek + English RAG reader: it answers strictly from the retrieved documents you put in its context, cites the documents it used as [n], and abstains when the answer is not there. It is the generation stage of a Greek RAG stack whose retrieval stages are Sophea-Nemo-Embedding and Sophea-Nemo-Reranker.

Creator: Kiefer SA
Base model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (NemotronHForCausalLM, hybrid Mamba + attention MoE, 30B total / A3B active; trust_remote_code)
Languages: Greek + English
Behaviour: grounded answer → [n] citation → abstain when unsupported
Training: LoRA (r 16, α 32) on attention + MLP + Mamba in_proj, 1 epoch, 8k context, assistant-only loss, bf16, cosine LR 1e-4, grad-clip 0.3, 4×B200 — merged into the base weights, so this is a standalone checkpoint (no adapter needed)

The base model is not a grounded reader. Off the shelf it answers from parametric memory: it hallucinates instead of abstaining (abstains on ~1% of unanswerable questions) and its answers are faithful to the context only a quarter of the time. This fine-tune teaches grounding, citation and selective prediction.

Why it matters — base vs fine-tuned

Measured on a Greek Wikipedia long-retrieval RAG benchmark (4,946 questions, up to 40 documents / ~24k tokens of context, 25% deliberately unanswerable, L1 factoid → L5 multi-hop synthesis). Identical prompts and greedy decoding for both models; correctness and faithfulness are LLM-judged.

Read. Correctness 29.4% → 66.9% and faithfulness 25.2% → 84.5% — the fine-tune answers from the documents instead of from memory. Citation F1 rises 41.1 → 67.8 and the model returns the exact gold citation set on 56.5% of items (base 30.1%). Abstention recall goes from ~zero to 30.5%: the base essentially never refuses, so its "faithfulness" on unanswerable questions is accidental.

Robustness to retrieval noise

More retrieved documents = more distractors. Correct / faithful (%), by number of documents in context:

Read. The fine-tune leads at every context size, from 76.5% correct at 8 documents to 59.3% at 40 — it degrades gracefully as noise grows, while faithfulness stays ≥79% throughout. The base is flat at ~29% because it is not reading the context in the first place.

Position of the gold document ("lost in the middle")

Read. The base is best when the gold document is early (31.9%) and worst when it is late (26.1%) — the classic position bias. The fine-tune is strongest on late gold (70.3%) and varies by only ~5 points across positions.

By question difficulty (L1 factoid → L5 multi-hop synthesis)

Read. The gain holds at every level — largest on the retrieval-shaped levels (L1 +43.8, L2 +45.3) and still +22 on the hardest multi-hop synthesis questions, where faithfulness rises from ~20-24% to ~69-70%.

Domain RAG evaluation

A second, independent evaluation on in-house Greek + English domain documents (800 questions across energy, finance, legal and medical; 187 of them unanswerable):

<div style="overflow-x:auto"> <table style="width:100%;border-collapse:collapse;font-size:13px;font-variant-numeric:tabular-nums;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif"> <thead><tr> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:left;">metric</th> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:center;">base</th> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:center;">fine-tuned</th> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:center;">Δ</th> </tr></thead><tbody> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Answer correctness (LLM-judge)</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">31.2%</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">61.0%</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">+29.8</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Faithfulness / groundedness</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">23.0%</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">66.2%</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">+43.2</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Citation hit (cited a gold document)</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">40.0%</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">60.1%</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">+20.1</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">Abstention on unanswerable items (n=187)</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">8.6%</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">80.2%</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">+71.6</td></tr> </tbody></table></div> <div style="overflow-x:auto"> <table style="width:100%;border-collapse:collapse;font-size:13px;font-variant-numeric:tabular-nums;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif"> <thead><tr> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:left;">metric</th> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:center;">Energy</th> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:center;">Finance</th> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:center;">Legal</th> <th style="padding:10px 8px;font-weight:500;border-bottom:2px solid #0a8043;color:#0a8043;font-size:14px;text-align:center;">Medical</th> </tr></thead><tbody> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">correct — base</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">37.5</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">22.5</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">42.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">23.0</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">correct — fine-tuned</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">62.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">57.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">72.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">53.0</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">faithful — base</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">25.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">21.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">23.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">23.0</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">faithful — fine-tuned</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">67.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">64.5</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">70.5</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">63.0</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);">cite-hit — base</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">35.5</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">40.5</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">44.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);">40.0</td></tr> <tr><td style="padding:7px 8px;padding-left:20px;text-align:left;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">cite-hit — fine-tuned</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">60.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">57.5</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">71.0</td><td style="padding:7px 8px;text-align:center;border-bottom:1px solid rgba(128,128,128,.15);font-weight:700;color:#0a8043;">52.0</td></tr> </tbody></table></div>

Read. The fine-tune wins on every domain and every metric, strongest on legal (72.0% correct / 70.5% faithful) and weakest on medical (53.0% / 63.0%) — the same ordering the retrieval stages show on these domains.

Selective prediction

On the 187 unanswerable items the correct action is to refuse. The base answers anyway 91% of the time; the fine-tune abstains on 80.2% of them — this is the mechanism behind the faithfulness gain.

Position invariance (answerable subset, n=613)

Read. The base decays as the gold document moves deeper into the context (45.4% at position 1 → 32.4% at position 6). The fine-tune is essentially flat at 77-85% across all six positions.

Usage (transformers)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

name = "KIEFERSA/Sophea-RAG-Nemo3"
tok = AutoTokenizer.from_pretrained(name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    name, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True).eval()

SYSTEM = (
    "Είσαι βοηθός που απαντά ΑΠΟΚΛΕΙΣΤΙΚΑ με βάση τα παρεχόμενα έγγραφα. Χρησιμοποίησε μόνο τις πληροφορίες των εγγράφων, ΜΗΝ επινοείς. Ανέφερε σε αγκύλες τον αριθμό του εγγράφου/των εγγράφων που χρησιμοποίησες, π.χ. [2]. Αν η απάντηση ΔΕΝ υπάρχει στα έγγραφα, πες το καθαρά. Απάντησε στα ελληνικά, σύντομα και ουσιαστικά."
)

docs = ["Η Αθήνα είναι η πρωτεύουσα της Ελλάδας.", "Το Παρίσι είναι η πρωτεύουσα της Γαλλίας."]
question = "Ποια είναι η πρωτεύουσα της Ελλάδας;"
context = "\n".join(f"[{i}] {d}" for i, d in enumerate(docs, 1))

msgs = [{"role": "system", "content": SYSTEM},
        {"role": "user", "content": f"Έγγραφα:\n{context}\n\nΕρώτηση: {question}"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
enc = tok(text, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**enc, max_new_tokens=200, do_sample=False)
print(tok.decode(out[0][enc.input_ids.shape[1]:], skip_special_tokens=True))
# -> "Η Αθήνα. [1]"

Number the documents [1..n] in the user turn and keep the system message above — the model was trained to cite those numbers and to say the answer is absent when it is. Greedy decoding (do_sample=False) is what the numbers above were measured with. Needs trust_remote_code=True.

Serving with vLLM

vllm serve KIEFERSA/Sophea-RAG-Nemo3 --trust-remote-code --max-model-len 40960

curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "KIEFERSA/Sophea-RAG-Nemo3",
  "temperature": 0,
  "messages": [
    {"role": "system", "content": "Απάντησε μόνο από τα έγγραφα και ανέφερε [n]."},
    {"role": "user", "content": "Έγγραφα:\n[1] Η Αθήνα είναι η πρωτεύουσα της Ελλάδας.\n\nΕρώτηση: Ποια είναι η πρωτεύουσα της Ελλάδας;"}
  ]
}'

License

Fine-tune of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 — governed by the NVIDIA Open Model License; commercial use permitted under those terms (note: not Apache-2.0). Review the base model's license before use.

Task: Text Generation
Library: transformers
Type: nemotron_h
License: other
Languages: el, en
Created: Jul 24, 2026
Updated: Jul 24, 2026