mist-qg-1.5b

mist-qg-1.5b

A compact multilingual question generator across 25 languages. Given a passage, it produces natural, search-style questions that the passage directly answers — the model is dual-use: a sellable /v1/generate-questions endpoint, and the data factory that mints (query, positive) training pairs for retriever and reranker fine-tuning, including several African languages underserved by existing tools. At ~1.5B parameters it runs comfortably on a single modest GPU.

📄 Model details

Training: fine-tuned on olaverse/qg-passages-multi (~50k passages, ~150k questions) distilled from CohereLabs/aya_collection_language_split via Qwen/Qwen2.5-32B-Instruct, with each generated question verified by round-trip retrieval before being kept for training (a question is discarded unless it retrieves its own source passage out of [source + distractors], embedded with Qwen/Qwen3-Embedding-0.6B).

🏃 How to run

Install transformers:

pip install -U transformers

The model expects a system + user message pair and returns strict JSON:

import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "olaverse/mist-qg-1.5b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

passage = "Tides are caused by the gravitational pull of the moon and, to a lesser extent, the sun, acting on Earth's oceans."
n, language = 3, "English"

messages = [
    {"role": "system", "content": "You write search-style questions that a passage directly answers."},
    {"role": "user", "content": f'''You are given a passage. Write {n} questions that the passage directly answers.

Rules:
- Each question must be answerable using ONLY this passage.
- Vary the type: factual, yes/no, and a comparison or "why/how".
- Natural, like a real user search query. Do NOT write "according to the passage".
- Write the questions in {language}.

Return ONLY JSON: {{"questions": ["...", "...", "..."]}}

Passage: {passage}'''},
]

input_ids = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

out = model.generate(input_ids, max_new_tokens=200, do_sample=False,
                     pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id)
text = tokenizer.decode(out[0][input_ids.shape[1]:], skip_special_tokens=True)
questions = json.loads(text[text.index("{"): text.rindex("}") + 1])["questions"]
print(questions)
# ["What causes ocean tides?", "Does the sun affect tides?",
#  "Which has a bigger effect on tides, the moon or the sun?"]

For production serving, wrap the same prompt behind vLLM with guided JSON decoding so the output is structurally guaranteed valid, not just usually valid.

📈 Performance

Round-trip keep-rate on 625 passages held out from training (never seen during fine-tuning): a generated question counts as "kept" if it retrieves its own source passage out of a pool of distractors (top-1), embedded with Qwen/Qwen3-Embedding-4B. This is an in-house benchmark (olaverse/qg-eval-multi-fresh), not a third-party/standardized one.

High-resource languages cluster at 0.95–1.00; the model's weakest languages are Amharic, Somali, and Shona (0.58–0.70) — treat outputs in these three with lower confidence than the rest of the set. This gap tracks limited fine-tuning data volume (~2,000 source passages/language) more than a fixed model-capacity ceiling, and is a natural target for a future data-scaling pass.

License

Released under Apache-2.0.

Citation

@misc{mist-qg-1.5b,
  title  = {mist-qg-1.5b},
  author = {Olaverse},
  year   = {2026},
  url    = {https://huggingface.co/olaverse/mist-qg-1.5b}
}