Amaretto-8B

Amaretto 8B

Amaretto 8B is a fine-tuned language model by AmarettoLabs, built for literary writing and writing assistance. It is fine-tuned from Ministral 3 8B Instruct using QLoRA.

The Amaretto line is designed to run on everyday hardware. Amaretto 8B is the mid-size model in the family — stronger creative writing quality than Amaretto 3B, while still practical for local and edge deployment.

Website: amaretto.ai · Contact: [email protected]

---

What it's good at

• Literary prose — scene writing, character voice, emotional interiority
• Writing assistance — editing, tone adjustment, conciseness, clarity, line editing
• Continuation — picking up a passage and extending it in the same register
• Style transfer — shifting a piece between registers (formal ↔ casual, journalistic ↔ personal)

---

Intended use

Amaretto 8B is a general-purpose writing companion. It works out of the box with no system prompt. For the strongest literary output, you can use the system prompt it was trained with:

You are a skilled literary fiction writer. Write with precision,
emotional depth, and a strong sense of voice. Favor showing over telling.
Use plain, direct language — avoid similes, forced comparisons, and
decorative imagery.

You can replace or extend this prompt for your own use case.

---

Usage

With llama.cpp / Ollama (GGUF)

GGUF quantized versions are available in this repository:

llama-cli -m amaretto-8b-Q8_0.gguf \
  --ctx-size 8192 \
  -sys "You are a skilled literary fiction writer." \
  -p "Write the opening of a scene where a woman returns to her childhood home."

Amaretto 8B inherits its base model's full context window via YaRN RoPE scaling, though fine-tuning was done at a 2,048-token sequence length — long-context generation quality on creative-writing tasks has not been specifically validated. --ctx-size above can be raised as needed.

Sampling: --temp 0.8 --min-p 0.1 works well for creative writing. min-p sampling is recommended over top-k/top-p — it noticeably reduces token-level glitches on small models.

With transformers

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "AmarettoLabs/Amaretto-8B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

messages = [
    {"role": "system", "content": "You are a skilled literary fiction writer. Write with precision, emotional depth, and a strong sense of voice."},
    {"role": "user", "content": "Write the opening of a scene where a woman returns to her childhood home after many years away."},
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
output = model.generate(input_ids, max_new_tokens=512, temperature=0.8, do_sample=True)
print(tokenizer.decode(output[0][input_ids.shape[-1]:], skip_special_tokens=True))

---

Training details

Training data consists of semi-synthetic and full-synthetic literary prose examples, writing assistance examples (editing, tone, clarity, continuation).

---

Limitations

• Optimized for English prose; other languages are not a focus
• Outputs can reflect biases present in the base model and training data
• The model may occasionally ignore system prompt instructions on complex or conflicting requests

---

About AmarettoLabs

AmarettoLabs is a small independent group based in California, USA, specializing in small language models and AI on the edge. The Amaretto line is released free and open.

Website: amaretto.ai · Contact: [email protected]

---

License

This model is released under the Apache 2.0 License, inherited from the base model. Base model: Ministral 3 8B by Mistral AI.