minilm-l6-option-head-doom

minilm-l6-option-head-doom

A 445k-parameter adapter that turns a frozen sentence encoder into a policy: it reads a situation as text, reads a list of things that could be done right now as text, and returns a probability for each.

It is not a generator. It never writes an action - it scores the ones it is handed, and the list may be different at every decision.

What it needs

sentence-transformers/all-MiniLM-L6-v2, frozen. This file is 1.7 MB of head weights and means nothing without it.

How the input is laid out

One packed sequence for the encoder:

rows 0..R    the situation, in windows of `max_span` tokens with overlap
rows R..R+S  one span per option, each "<instructions> [SEP] <option text>"

Each option span starts with [CLS]. The head reads those rows as queries, cross-attends over the situation's rows, and emits one scalar per option. The softmax is taken on the host, over that call's options only - which is why the number of options is free to change between calls and no kernel has an opinion about it.

q   = hidden[cls_rows] @ Wq  + bq      [S, 384]
kv  = hidden[0..R]     @ Wkv + bkv     [R, 768]
ctx = crossattn(q -> kv)               [S, 384]
out = LayerNorm(ctx + q)               [S, 384]
z_i = out_i . w + b                    [S]
p   = softmax(z)                       host side

Tensors

head.wq.weight    [384, 384]     head.wkv.weight   [768, 384]
head.wq.bias      [384]          head.wkv.bias     [768]
head.ln.weight    [384]          head.ln.bias      [384]
head.score.weight [1, 384]       head.score.bias   [1]

What this particular head was trained on

DOOM, through an engine that hands the agent a text observation and a list of options each decision. Behaviour cloning from a scripted player, then PPO.

It matches its teacher and does not beat it. Over 24 shared episodes:

             exits  deaths  progress
scripted        10       5      0.70
this policy     10       2      0.73

Twelve iterations of policy gradient bought what the behaviour cloning had already reached. Read it as a demonstration of the architecture, not as a strong DOOM player.

Limitation worth knowing

Each option's score depends only on that option and the situation - options are coupled solely by the softmax that normalises them. There is no mechanism for comparing two options directly, so any preference between them has to be inferable from the situation text.

---

Swedish Embedded AB builds decision models that run on the customer's own hardware, choosing among actions a machine defines at run time. If your team needs that, you can procure our services by sending an email to [email protected].