Nova-70B-Llama-3.3

Nova-70B-Llama-3.3

Built on Llama 70B and trained with the same techniques that made Muse good at stories about relationships and character development, Nova brings the greater reasoning abilities of a larger model to understanding the nuance that makes characters feel real and stories come to life. Whether you're roleplaying cloak-and-dagger intrigue, personal drama or an epic quest, Nova is designed to keep characters consistent across extended contexts while delivering the nuanced character work that defines compelling stories.

If you want to try this model without running it yourself, you can do so at https://aidungeon.com.

We plan to continue improving and open-sourcing similar models, so please share any and all feedback on how we can improve model behavior. Below we share more details on how Nova was created.

Quantized GGUF weights can be downloaded here.

Model details

Nova 70B was trained directly on top of Llama 3.3 70B Instruct with a multitude of datasets, combining text adventures of the kind used to finetune our Wayfarer model series, long emotional narratives, detailed worldbuilding and general roleplay, each carefully balanced and rewritten to be free of common AI cliches. A small single-turn instruct dataset was included to send a stronger signal during finetuning.

DPO was explored in this particular size range since this had a noticeable impact on Muse’s more human writing style but combined with Llama 3.3’s own DPO training this produced a noticeably worse performing model.

Inference

The Llama architecture can handle a large variety of settings, with the following serving as a baseline:

"temperature": 0.8,
"repetition_penalty": 1.05,
"min_p": 0.05

Limitations

Much like many of our other models, Nova was trained exclusively on second-person present tense data (using “you”) in a narrative style, but the larger model size and the underlying instruct layer make it more then capable of producing narrative texts with other perspectives while retaining its ability to understand complex instructions.

Prompt Format

This model follows the original Llama 3 prompt format.

<|start_header_id|>system<|end_header_id|>

You're a masterful storyteller and gamemaster. Write in second person present tense (You are), crafting vivid, engaging narratives with authority and confidence.<|eot_id|>
<|start_header_id|>user<|end_header_id|>

> You peer into the darkness.<|eot_id|>
<|start_header_id|>assistant<|end_header_id|>

You have been eaten by a grue.

GAME OVER<|eot_id|>

Credits

Thanks to Gryphe Padar for collaborating on this finetune with us!