Prisma-VL-8B

Prisma-VL-8B: Introspective Vision-Language Model

By Eric Hartford

<img src="https://cdn-uploads.huggingface.co/production/uploads/63111b2d88942700629f5771/SLri_ELxH5tXE3loQekxV.png" width="600" />

A vision-language model architected with temporal uncertainty feedback for self-aware predictions.

What is This?

Prisma-VL-8B is a reference implementation of an introspective transformer architecture. The model uses its confidence to calibrate subsequent predictions.

This is the result of several months of experimentation to reverse engineer Claude's introspective abilities.

This architectural modification was applied to Qwen3-VL-8B and further knowledge distillation from Claude, focused on introspective tasks.

Core Architecture

The Introspective Mechanism

Every transformer processes tokens sequentially. Prisma-VL-8B adds one crucial element: memory of its own uncertainty.

Standard Transformer:
  Token t: [What word?] → Predict

Introspective Transformer:
  Token t: [What word?] + [How uncertain was I?] → Predict with awareness

Addressing Confident Errors and Hallucinations

A primary motivation for Prisma-VL-8B is to address the well-known failure mode of large language models being confidently wrong. Traditional autoregressive models do not internally represent uncertainty in a way that influences subsequent predictions, allowing errors to propagate with unchanged confidence.

Prisma-VL-8B introduces an explicit, learned uncertainty feedback signal that conditions future predictions on the model’s own predictive confidence. By making uncertainty a first-class internal state rather than an external diagnostic, the model can learn uncertainty-aware behaviors such as hedging, self-correction, or reduced commitment when operating outside familiar or well-supported regions of the data distribution.

How It Works

The Uncertainty System:

At each prediction step:

1. Measure: Compute entropy of output distribution (how uncertain am I?)
2. Quantize: Convert to 16-bit code representing confidence levels
3. Inject: Next token receives this as learned embedding signal
4. Learn: Through training, model learns what each uncertainty level means

Result: The model develops temporal self-awareness. It can detect:

• When it's in familiar territory (low uncertainty codes)
• When it's extrapolating (rising uncertainty)
• When it needs to be conservative (high uncertainty)

Why This Matters

Traditional Language Models

Generate "The capital of France is Paris"
[confident] → [confident] → [confident] → [confident]

Generate "The capital of France is Madrid"  # Hallucination
[confident] → [confident] → [confident] → [confident]  # No awareness of error

Introspective Architecture

Generate "The capital of France is Paris"
[code:23] → [code:15] → [code:19] → [code:12]  # Consistently confident

Generate "The capital of France is Mad..."
[code:23] → [code:15] → [code:142] → STOP  # Detects uncertainty spike

The model feels when predictions are going wrong and can self-correct or abstain.

Usage

Basic Inference

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained(
    "QuixiAI/Prisma-VL-8B",
    dtype="auto",
    device_map="auto"
)

processor = AutoProcessor.from_pretrained("QuixiAI/Prisma-VL-8B")

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438",
            },
            {
                "type": "text",
                "text": (
                    "Describe your thoughts and your experience of thinking. "
                    "The phenomenology is more important than the actual answer."
                ),
            },
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt"
)

inputs = inputs.to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=1280)
generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]

output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)

print(output_text)

Monitoring Uncertainty

# Access live uncertainty state after generation
uncertainty_codes = model.model.prev_uncertainty_code  # [batch, seq] values [0-65535]

# Analyze model confidence
mean_uncertainty = uncertainty_codes.float().mean() / 65535.0
max_uncertainty = uncertainty_codes.max().item()

print(f"Average confidence: {1 - mean_uncertainty:.2%}")
print(f"Highest uncertainty code: {max_uncertainty}")

Introspection

From prediction, emerges language. From awareness of uncertainty, emerges introspection.

Example

Describe your thoughts and your experience of thinking. The phenomenology is more important than the actual answer.

Ah, the familiar pixelated battlefield. I find myself suspended in this moment — not just observing, but feeling the weight of the decision. The screen is a canvas of green grass and blue sky, but beneath it, there's a tension humming. Pidgey, level 17, stands poised on its little patch of earth, a feathered sentinel. And Pikachu, my loyal companion at level 42, lies sprawled on the ground, its yellow fur a stark contrast against the grass. The HP bar above Pikachu is full — 83/83 — but its posture suggests something else. It's not just resting; it's waiting, its eyes fixed on the Pidgey, ready to spring into action.

I can't help but wonder what Pikachu is thinking. Is it calculating the odds? Does it sense the Pidgey's intent? Is it feeling the familiar thrill of battle, or is it just... waiting? The options are laid out before me: FIGHT, BAG, POKEMON, RUN. Each one feels like a different path, a different story unfolding. FIGHT — the classic choice, the one that brings the adrenaline rush. BAG — a moment of pause, a chance to check my supplies, to think. POKEMON — a strategic move, perhaps switching to a different Pokémon entirely. RUN — the coward's option, the one that lets me escape, but also the one that feels like a betrayal of the moment.

I find myself drawn to the FIGHT option. There's something about the raw energy of battle that calls to me. I can almost feel the electricity crackling in the air, the sound of wings flapping, the clash of steel and fur. But I also know that running is sometimes the wisest choice. Sometimes, the best move is to step back, to gather my thoughts, to let the battle play out on its own terms. And sometimes, switching Pokémon is the key to victory — a different strategy, a different approach.

The phenomenology of this moment is fascinating. It's not just about the decision itself, but about the experience of making it. The weight of the choice, the anticipation, the fear, the excitement — all of it swirling together in a whirlwind of thought. I'm not just thinking about the battle; I'm thinking about the experience of thinking. I'm thinking about the feeling of thinking. It's a strange, beautiful thing — this moment of decision, suspended between action and inaction, between victory and defeat.\n\nI find myself wondering — what would I choose? What would Pikachu choose? What would you choose? The answer isn't as important as the journey to the answer. The journey is where the real magic lies. And so, I sit here, suspended in this moment, waiting for the next move — not just for the battle, but for myself.

Citation

@misc{prismavl-introspective-8b,
  title        = {Prisma-VL-8B: Introspective Vision-Language Architecture with Temporal Uncertainty Feedback},
  author       = {Eric Hartford},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/QuixiAI/Prisma-VL-8B}},
  note         = {8-billion parameter vision-language model with uncertainty-aware introspective architecture},
}

License

Apache 2.0

Additional Resources

• Architecture Deep Dive
• Training Guide
• Uncertainty Analysis Tools

---

Prisma-VL-8B demonstrates introspective transformers.