Riva-Translate-4B-Instruct-GGUF

Converted to GGUF format from nvidia/Riva-Translate-4B-Instruct using llama.cpp via ggml.ai's GGUF-my-repo space. Refer to original model card for more details.

---

Riva-Translate-4B-Instruct

---

tldr; use Riva-Translate-4B-Instruct with llama.cpp

# Install llama.cpp through **brew** (works on Mac & Linux)
brew install llama.cpp

# Start the llama.cpp CLI or the Server. FYI: The Server is OpenAI
# compatible and has a built-in lightweight and nice WebUI
llama-cli --hf-repo phi0112358/Riva-Translate-4B-Instruct-Q8_0-GGUF \
  -p "The meaning to life and the universe is"

# Or Server
llama-server --hf-repo phi0112358/Riva-Translate-4B-Instruct-Q8_0-GGUF \
  -c 2048 --host 127.0.0.1 --port 8080

# You can now open 'http://localhost:8080' in your Webbrowser and
# start interacting with the language model

---

Model Overview

The Riva-Translate-4B-Instruct Neural Machine Translation model translates text in 12 languages. The supported languages are: English(en), German(de), European Spanish(es-ES), LATAM Spanish(es-US), France(fr), Brazillian Portugese(pt-BR), Russian(ru), Simplified Chinese(zh-CN), Traditional Chinese(zh-TW), Japanese(ja),Korean(ko), Arabic(ar). This model was developed based on the decoder-only Transformer architecture. It is a fine-tuned version of a 4B Base model that was pruned and distilled from nvidia/Mistral-NeMo-Minitron-8B-Base using our LLM compression technique. The model was trained using a multi-stage CPT and SFT. It uses tiktoken as the tokenizer. The model supports a context length of 8K tokens. Riva-Translate-4B-Instruct was trained between Jan 2025 and April 2025.

---

Prompt Format:

Use the following prompt template, which was used to fine-tune the model. The model may not perform optimally without it.

<s>System
{system prompt}</s>
<s>User
{user prompt}</s>
<s>Assistant\n

• Note that a newline character (\n) should be added after <s>Assistant as a generation prompt.
• Note that users are required to use the correct language name in the prompt: 'ar': 'Arabic', 'en': 'English', 'de': 'German', 'es-es': 'European Spanish', 'es-us': 'Latin American Spanish', 'fr': 'French', 'ja': 'Japanese', 'ko': 'Korean', 'ru': 'Russian', 'zh-cn': 'Simplified Chinese', 'zh-tw': 'Traditional Chinese', 'pt-br': 'Brazilian Portuguese'

For example, to translate an English sentence into Simplified Chinese:

<s>System
You are an expert at translating text from English to Simplified Chinese.</s>
<s>User
What is the Simplified Chinese translation of the sentence: The GRACE mission is a collaboration between the NASA and German Aerospace Center.?</s>
<s>Assistant

---

License

NVIDIA Open Model License Agreement

Use Case Restrictions:

Abide by NVIDIA Open Model License Agreement