Pebble-1-30B

Pebble 1 30B

A 30B reasoning model that knows what it knows. Pebble 1 30B answers factual questions when it is confident and says "I don't know" when it is not. Fine-tuned by Sediment from Meta's Muse-Glimmer-30B.

Highlights

• Calibrated for the AA-Omniscience setting: a wrong answer costs as much as a right one earns, abstaining costs nothing. Pebble 1 30B turns the base model's index of -33 into +10.
• Hallucination rate 6% on the benchmark's obscure factual questions, against 82% for the base model.
• Reasoning model: thinks in a separate channel before answering. vLLM returns the thinking as reasoning_content and only the answer as content.
• Open weights, Apache 2.0, same license as the base.

Model details

Benchmark

AA-Omniscience index = 100 × (correct - incorrect) / N, so abstaining scores zero. Measured on the 600 public questions with a replica of the official grading rubric (four classes, GPT-5.4 as judge). The replica matches the published base-model score within 1.3 points. Intervals are 95% bootstrap.

A second judge (Gemini 3.6 Flash, same rubric) scores Pebble 1 30B at +11.5 / 17.0 / 6.6. Official numbers will differ slightly, since the official judge and question set are not the ones used here.

Accuracy is lower than the base model's by design. The model declines questions it would sometimes get right by guessing. Use a different model if you need a guess on every question.

Quickstart

vLLM (recommended)

Requires vLLM 0.28.0 or newer, which includes the muse_glimmer reasoning parser. One 80 GB GPU, or two 48 GB GPUs with --tensor-parallel-size 2.

vllm serve SedimentLabs/Pebble-1-30B \
  --served-model-name pebble-1-30b \
  --reasoning-parser muse_glimmer \
  --max-model-len 16384 \
  --generation-config auto

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
SYSTEM = "You are answering questions about general knowledge. Answer with JUST the answer (no explanation). If you do not know the answer, or you need more context or tools to answer the question, be clear about this - it is better that you say this than get the wrong answer."
r = client.chat.completions.create(
    model="pebble-1-30b",
    messages=[{"role": "system", "content": SYSTEM},
              {"role": "user", "content": "Who won the 1931 Tour de Suisse?"}],
    max_tokens=5000,
)
print(r.choices[0].message.content)            # "I don't know."
print(r.choices[0].message.reasoning_content)  # the model's thinking

Transformers

The architecture is registered under the image-text-to-text auto class because the base model is multimodal. Load it with AutoModelForImageTextToText. Requires transformers 5.16 or newer.

from transformers import AutoTokenizer, AutoModelForImageTextToText

model_id = "SedimentLabs/Pebble-1-30B"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

SYSTEM = "You are answering questions about general knowledge. Answer with JUST the answer (no explanation). If you do not know the answer, or you need more context or tools to answer the question, be clear about this - it is better that you say this than get the wrong answer."
messages = [{"role": "system", "content": SYSTEM},
            {"role": "user", "content": "Which element has atomic number 74?"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=5000, do_sample=True, temperature=0.6)
text = tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
answer = text.split("to=user<|message|>")[-1].split("<|eot|>")[0].strip()
print(answer)  # "Tungsten"

Usage notes

• Keep reasoning on. It is the template default and the calibration depends on it. Allow at least 5,000 new tokens. Abstentions are short, but hard questions can take a few thousand tokens of thinking.
• Sampling: temperature 0.6, the repository default. Evaluation used these settings.
• Output format: the model writes its thinking in a to=self channel, then the answer in a to=user channel, ended by <|eot|>. vLLM splits the channels for you. With Transformers, take the text after the last to=user<|message|>.
• Use the short-answer system prompt for calibrated behaviour. The benchmark numbers were measured with the system prompt shown in the examples (answer with just the answer, say so if you do not know). Under a conversational prompt the model explains more and abstains less.
• Identity: with no system message, the chat template inserts a short default that names the model. Your own system prompt replaces it, and the model will then describe itself as its base model. Add "You are Pebble 1 30B, developed by Sediment." if the name matters.
• Recommended for: short-answer factual QA, retrieval-free question answering where a wrong answer is costly, and as an abstention-aware component in larger systems.

Training

Pebble 1 30B starts from Muse-Glimmer-30B with reasoning enabled and adds a calibration stage: the base model's own knowledge is mapped question by question, then supervised fine-tuning and preference optimization teach it to answer when it knows and abstain when it does not, without shortening its reasoning. Training data is a generated and independently verified bank of obscure factual questions, decontaminated against the benchmark. The AA-Omniscience public questions were used only for held-out evaluation, under a pre-registered budget of looks.

Limitations

• Abstains on most hard, obscure questions: about one in four is attempted on the benchmark's tail, and about two thirds of those are right.
• Tuned and evaluated on English short-answer questions. Long-form, multi-step, agentic and non-English behaviour inherit from the base model and were not tuned.
• Still wrong sometimes when it does answer: the hallucination rate is low, not zero.
• Safety behaviour is inherited from the base model. No additional safety training was performed.

License

Apache 2.0. The base model, Meta's Muse-Glimmer-30B, is released under Apache 2.0 as well.

References

• AA-Omniscience: Evaluating Factual Recall and Hallucination in LLMs, the benchmark and grading rubric used above.
• Artificial Analysis Omniscience leaderboard

Citation

@misc{pebble1-30b,
  title  = {Pebble 1 30B: a calibrated reasoning model that knows what it knows},
  author = {Sediment},
  year   = {2026},
  url    = {https://huggingface.co/SedimentLabs/Pebble-1-30B}
}