openjev

OpenJev

Typed decisions about text, web pages and screenshots. One forward pass per question with up to 52 options. About 210 ms for an isolated web decision on one H100, with roughly 1,460 prompt tokens and 23 options.

No free-form text to parse. No chain of thought. No training per task.

OpenJev is an open-weights decision model. You describe the decision in plain words at request time, with your own labels, and it answers with a choice, a yes / no probability or a score. The same model routes a support ticket, flags a policy violation, judges whether an answer is grounded, and tells a browser agent which button to press next.

**

curl -s http://localhost:3000/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "openjev",
  "state": "Customer message: I was charged twice for my order last week and nobody has replied.",
  "questions": {
    "route":   {"type": "choice", "instructions": "Which team should handle this?",
                "criteria": {"billing": null, "shipping": null, "technical": null}},
    "angry":   {"type": "noul",   "instructions": "Is the customer angry?"},
    "urgency": {"type": "score",  "instructions": "How urgent is this?",
                "criteria": ["can wait", "this week", "today", "right now"]}
  }
}'

Three questions in one API request, three typed answers, each with a probability for every option. One forward pass per question (up to 52 options).

Why it is different

• Labels live in the request. No labelled data, no training run, no fixed label set. New task, new labels, new domain: change the JSON, not the model.
• Not a chatbot you have to parse. The answer is read from the scores at the first output position: one score per option, converted to probabilities using fixed calibration settings. There is no free-form answer to parse and no long generation to wait for.
• Fast enough to sit inside a loop. About 80 ms for a short text decision and about 210 ms for an isolated web decision with roughly 1,460 prompt tokens and 23 candidate elements, on one H100.
• Built for agents as well as text. It reads DOM, JSON and screenshots, picks the target element, picks the next operation and says when the task is finished.
• Stable. Shuffle the option order and the answer changes in 2.3% of cases (18.5% for the same model before tuning).

Use cases

Results

How these were measured: every test set was held out from our fine-tuning (checked by id and by content against our training sets). Of the 10,000 text questions, 6,922 were fresh for this run and 3,078 had been used during development; the tables show both together and the fresh-only gap is given below. In the text tables the hosted Jev API answered exactly the same questions; the agent, language and screenshot rows compare OpenJev with its own base model.

Text decisions: 10,000 text questions, 34 public sources, the same questions for every model

Within 2 points of the hosted Jev API (1.4 points behind; 1.2 on the 6,922 fresh additions and replacements), 3.7 points above its own base, 8.3 points ahead of Nimble 9B, with open weights you can run yourself. Every question has at most 26 options and fits Nimble's 2,048-token limit, so all four models received the same state, question text, options and option order. Only the hosted model-routing field differed. There was one attempt per question, with no retries. Three invalid or failed hosted responses (one HTTP 520 error, two answers whose chosen option was not their own highest-probability option) count as incorrect; the other arms had none.

The legal rows include 613 distinct merger-agreement clauses from 151 contracts, scored with native expert annotations: OpenJev 76.7%, hosted 73.7%, before tuning 75.4%. These are held-out contract-question pairs, not unseen contracts.

Agent decisions, languages and text: before and after tuning

Whole browser tasks, end to end

OpenJev ties the hosted Jev API on our matched development run. 100 MiniWoB task types, one attempt each, the same open text-only client for every model. About a quarter of these tasks need colour, shapes, drawing or dragging, which a text-only click-and-type client cannot do, so no model gets close to 100 here; what matters is the comparison on identical conditions.

Languages and long documents

Speed, one H100, FP8

• Short text decisions: about 80 ms each (3,347 sequential questions in about 4.5 minutes).
• Desktop step with a screenshot: 176 ms median. Web step (about 1,460 prompt tokens, about 23 options): about 210 ms median.
• 100 questions about one 4,100-token page: 3.9 s in total with helper queue concurrency 16, an average of 39 ms per question across the batch. This is not sequential request latency.
• 7 requests per second on distinct 1,200-token pages at concurrency 8, on a single GPU.

Quick start

pip install "vllm==0.29.0" "openai==3.16.2" "httpx==0.28.1"
hf download openjev/openjev --local-dir openjev

# 1. the model
vllm serve ./openjev --host 127.0.0.1 --served-model-name qwen --port 8000 \
  --enable-prefix-caching --max-model-len 16384 --gpu-memory-utilization 0.90 \
  --limit-mm-per-prompt '{"image":1}' --trust-remote-code --max-num-seqs 256 \
  --max-logprobs 64 --gdn-prefill-backend triton --quantization fp8

# 2. the decision API in front of it
VLLM=http://localhost:8000/v1 TOKENIZER=./openjev \
READOUT_T=0.85 READOUT_NOUL_T=1.829074 READOUT_NOUL_BIAS=0 \
READOUT_TARGETED=1 READOUT_INSTR_STYLE=pyrepr SHIM_STAGGER=1 \
python openjev/helper/shim.py --host 127.0.0.1 --port 3000

Then POST /v1/systemone as in the example at the top. Full serving guide, every flag and the security notes for exposing the port: serve/SERVE.md.

The request and response shapes follow the hosted Jev API, so a client written for it can be pointed at your own server.

Tip for agent builders: treat DONE as the model's opinion and confirm completion in your own loop (a success message, a changed URL, a saved record) before you stop.

Formats

Every quantized format is listed with its own measured accuracy on the same frozen questions.

Request limits

• Up to 52 options per question in a single pass. More options are handled in several passes (groups of 52, then a final pass over the winners).
• Prompts up to 16,384 tokens. One image per request.
• Probabilities are calibrated by the helper with the fixed settings shown in the quick start; keep them as they are to reproduce the numbers on this page.

How it works, in one paragraph

A language model computes a score for every possible next token before it writes anything. OpenJev gives each of your options a letter, asks the question, and reads the scores of exactly those letters at the first output position (the server is asked for a single token, never a free-form answer). For questions with up to 52 options, one forward pass produces one number per option, then a calibration step turns the numbers into probabilities you can threshold. The model is tuned so that this single position carries the decision: trained to select the correct option and stay consistent when the options are reordered.

Licence

OpenJev weights are released under CC BY-NC 4.0: free for research and other non-commercial use, with attribution. For commercial use, open a discussion on this repository.

The files in helper/ and serve/ are Apache 2.0. OpenJev is built on an Apache 2.0 open base model; the required attribution is in NOTICE and the Apache text is in LICENSE-APACHE-2.0.

OpenJev is an independent project, not affiliated with TypeSafe; Jev is their product.