Qwen3.5-0.8B-quotes-ONNX

Qwen3.5-0.8B-quotes-ONNX

A fine-tuned version of Qwen3.5-0.8B optimized for generating original, poetic motivational quotes. Converted to ONNX with Q4 quantization for efficient in-browser inference via Transformers.js.

Try it live: mentria.ai/tools/quote · Run an LLM in your browser

How it works

The model runs entirely in your browser using WebGPU (or WASM fallback), with no account or API key. Your prompts are processed on your device, and the weights are cached locally after the first download.

Usage with Transformers.js

import { AutoProcessor, AutoModelForImageTextToText, TextStreamer } from '@huggingface/transformers';

const MODEL_ID = 'mentriaai/Qwen3.5-0.8B-quotes-ONNX';

const processor = await AutoProcessor.from_pretrained(MODEL_ID);
const model = await AutoModelForImageTextToText.from_pretrained(MODEL_ID, {
  dtype: { embed_tokens: 'q4', vision_encoder: 'fp16', decoder_model_merged: 'q4' },
  device: 'webgpu',
});

const messages = [
  { role: 'system', content: 'You are a poetic thinker. Write one short, original motivational quote (1-2 sentences). Do not attribute it to anyone. No preamble, no quotation marks — just the quote itself.' },
  { role: 'user', content: [{ type: 'text', text: 'Give me a motivational quote.' }] },
];

const textContent = processor.apply_chat_template(messages, {
  tokenize: false,
  add_generation_prompt: true,
});

const inputs = await processor(textContent, null, { padding: true, truncation: true });

const streamer = new TextStreamer(processor.tokenizer, {
  skip_prompt: true,
  skip_special_tokens: true,
  callback_function(token) {
    process.stdout.write(token);
  },
});

await model.generate({
  ...inputs,
  max_new_tokens: 128,
  do_sample: true,
  temperature: 0.9,
  top_p: 0.95,
  streamer,
});

Example outputs

The seeds you plant in silence become the forests that speak for you.

Fear is a compass — it always points toward the thing worth doing.

You are both the sculptor and the marble — chip away everything that is not you.

The coastline of your dreams is where the sea meets the shore — follow it.

Model details

ONNX files

onnx/
  decoder_model_merged_q4.onnx       (813 KB graph + 452 MB weights)
  embed_tokens_q4.onnx               (857 B graph + 155 MB weights)
  vision_encoder_fp16.onnx           (183 KB graph + 195 MB weights)

Total download: ~802 MB on WebGPU (cached by the browser after first load).

Training details

• Learning rate 2e-4 (10x the standard full fine-tuning rate)
• All linear layers targeted (attention + MLP, not just attention)
• Prompt masking enabled (loss computed only on quote output tokens)
• 200 iterations selected from early stopping analysis to balance style learning vs. memorization

About mentria.ai

Mentria is AI that runs in your browser, with built-in tools, an extensions library and card decks for learning. Add your own extensions and decks. Your data stays private. The motivational quote generator is one of its AI tools: mentria.ai/tools/quote.