ST-Presets-Mistral-Small

This repo contains my preferred SillyTavern settings for Mistral Small models.

Mistral Small is a fantastic model, but its user/assistant turn structure means that the context/prompt composed by SillyTavern can mess up the instruct formatting for things like Lorebook/World Info injection or Author's Note injection, or result in repeated Assistant or User turns that can confuse the model.

I've had the best luck composing the context as a single user turn, asking the model to respond to what's essentially a very long single-turn prompt.

The composed context tends to look something like this:

<s>[INST] System prompt: Use the following information to write {{char}}'s next response in the Exchange below.

Character description
Persona description
Scenario description
World Info

Exchange:
{{char}}: First message</s>
{{user}}: First response
{{char}}: Second message</s>
{{user}}: Second response

Author's Note: Write 2 paragraphs using vivid and natural prose.[/INST] {{char}}:

This prevents instruct formatting issues from AN/WI insertions, and avoids problems with repeated turns depending on whether the first message in the chat is from the Character or the User.

When using AN with this template, it should work correctly with Before/After Story String insertion. For more immediate insertion, Depth 1 (as shown in the example above) is least likely to confuse the model as it doesn't interrupt the chat history.

Note: <s> is added automatically by the tokenizer and not included in the template. </s> is added after each Character message in the chat history to prevent problems with the AI not ending its turn properly when writing the newest response.