mini-Jev

mini-Jev

A Qwen3-0.6B-based decision model for tool selection, routing, and agent-control experiments.

mini-Jev is an experimental decision model designed to make structured control decisions inside an agent loop:

state + available candidates → probabilities + selected decision

The current release:

• Uses Qwen3-0.6B as a backbone.
• Keeps the Qwen backbone frozen.
• Adds a lightweight trained decision head (~1.1 MB) to score candidate actions directly.
• Performs candidate scoring and selection; it is not designed for text generation.
• Serves as an experimental baseline and demo for fast, structured agent decision-making.

Note on naming: The model implementation was originally developed under the internal name ODM Mini v1. That name remains in class names (ODMMiniModel), loader files (odm_mini.py), and config.json for backwards compatibility.

---

1. What is mini-Jev?

AI agents frequently encounter situations where they must make small, structured decisions rather than generate free-form text. Common examples include:

• Tool selection: Deciding which tool or external integration to invoke.
• Function routing: Selecting the specific API or method to execute next.
• Next-action prediction: Choosing between continuing investigation, asking for clarification, or concluding.
• Continue / finish decisions: Deciding whether an assigned task is satisfied or requires another step.
• Model routing: Directing incoming tasks to appropriate candidate downstream models.

Rather than invoking a large generative language model for every routing and control decision, mini-Jev explores using a small, specialized decision model to evaluate candidate options directly over agent state.

mini-Jev is an experimental baseline and is not intended for unmonitored production use.

---

2. Current model

Current release: Qwen3-0.6B-based mini-Jev v1

Architecture

Frozen Qwen3-0.6B → candidate representation → lightweight decision head → grouped softmax

1. Backbone: The base model is Qwen/Qwen3-0.6B, which remains completely frozen. The weights in this repository contain only the trained decision head; the Qwen backbone is downloaded separately by the loader.
2. Representation pooling: State text and candidate descriptions are passed through the model. Only candidate-description tokens are mean-pooled. Candidate IDs, prompt formatting headers, state tokens, and padding tokens are excluded from the pooling mask.
3. Decision head: A lightweight two-layer MLP:
   • Linear(1024, 256)
   • GELU
   • Linear(256, 1)
   • Total head parameters: 262,657 (~1.1 MB).
4. Scoring: The resulting scalar logits are normalized across candidate options using a grouped softmax to yield probabilities, a selected candidate, confidence, and decision margin.

---

3. Quick start

Installation

pip install torch transformers safetensors huggingface_hub

Download the loader module:

hf download samatv256/mini-Jev odm_mini.py --local-dir .

Basic usage

from odm_mini import ODMMiniModel

model = ODMMiniModel.from_pretrained("samatv256/mini-Jev")

choice = model.predict_choice(
    state={"user_request": "Find the weather in Boston."},
    candidates=[
        {
            "id": "weather.lookup",
            "description": "Look up the current weather.",
        },
        {
            "id": "calendar.list",
            "description": "List calendar events.",
        },
    ],
)

print("Selected:", choice.selected)
print("Probabilities:", choice.probabilities)

Full snapshot download and usage

You can also download all repository files to a local directory before loading:

import sys
from pathlib import Path
from huggingface_hub import snapshot_download

release_dir = Path(
    snapshot_download(
        repo_id="samatv256/mini-Jev",
        allow_patterns=[
            "README.md",
            "model.safetensors",
            "config.json",
            "odm_mini.py",
            "LICENSE",
        ],
    )
)
sys.path.insert(0, str(release_dir))

from odm_mini import ODMMiniModel

model = ODMMiniModel.from_pretrained(release_dir)

state = {
    "user_request": "Find the current weather in Boston.",
    "available_context": "The user has not provided weather data.",
}
candidates = [
    {
        "id": "weather.lookup",
        "description": "Look up current weather for a specified city.",
    },
    {
        "id": "calendar.list",
        "description": "List upcoming calendar events for the user.",
    },
    {
        "id": "control.finish",
        "description": "Finish because the request has already been completed.",
    },
]

choice = model.predict_choice(state=state, candidates=candidates)
print("Selected candidate:", choice.selected)
print("Probabilities:", choice.probabilities)
print("Confidence:", choice.confidence)
print("Decision margin:", choice.decision_margin)
print("Latency (ms):", choice.latency_ms)

ODMMiniModel.from_pretrained() accepts either the Hugging Face repo ID or a local directory path. CUDA runs in BF16 by default; CPU runs in FP32.

---

4. Example use cases

mini-Jev can be used for several structured agent-control patterns:

• Tool selection: Choosing the right tool from a list of available integrations.
• Function routing: Determining which endpoint or handler to call.
• Next-action prediction: Selecting intermediate steps in a reasoning or execution plan.
• Agent-control research: Investigating small-footprint models for local agent decision-making.
• Model-routing experiments: Evaluating routing policies between different candidate models. (Note: Model routing is a conceptual application of the candidate-selection interface, not a validated production routing capability.)

Conceptual example

State:
User wants to find the weather in Boston.

Candidates:
- web_search
- weather_tool
- calculator
- finish

Decision:
weather_tool

---

5. Jev Decisions v1

Researchers interested in training and evaluating decision models can explore the Jev Decisions v1 dataset.

• Dataset card: https://huggingface.co/datasets/samatv256/jev-decisions-v1
• Description: A public dataset containing nearly 12 million canonical agent-decision records (over 6.2M choice-eligible decisions across 1.06M task trajectories) with explicit candidate sets, chosen actions, and eligibility flags.

Important: The current mini-Jev Qwen3-0.6B baseline was NOT trained on Jev Decisions v1. It is provided as an open project resource for community research.

---

6. Current verified results

The results below reflect the public Qwen3-0.6B baseline checkpoint:

Training configuration

• Training data: 50,000 synthetic decision examples
• Backbone: Frozen Qwen/Qwen3-0.6B
• Trained components: DecisionHead weights only (grouped cross-entropy objective)
• Checkpoint: Seed 41, epoch 4
• Temperature: Fixed at 1.0; maximum softmax output is reported as confidence.

Synthetic held-out evaluation

• Latency measurements: On an NVIDIA GH200 using BF16 backbone inference and a shared-prefix KV-cache path, total latency was approximately 76–85 ms for contexts of 256–1,024 state tokens across 3–16 candidates (measured range: 76.11–82.36 ms). Candidate representation extraction reached 831.3 candidates/second at batch size 512.

Real shadow-agent evaluation

In an offline evaluation on real agent executions across 75 multi-step trajectories and 243 decisions, the baseline achieved:

• Action accuracy: 27.98%
• Controller agreement: 26.34%

These metrics are reported to transparently show the significant transfer gap between synthetic single-step decisions and dynamic, multi-step agent trajectories.

---

7. Limitations

mini-Jev v1 is an experimental research prototype and is not ready for unmonitored production agent control.

• Limited real-agent transfer: While performance on synthetic single-step benchmarks is moderate (~73%), accuracy drops substantially (~28%) in real multi-step agent environments.
• Premature completion: The model exhibits a known failure mode of high-confidence premature finish decisions. After making partial progress, it frequently over-indexes on successful intermediate receipts and chooses control.finish before completing remaining steps.
• Uncalibrated confidence: Grouped softmax probabilities indicate relative preference among the provided candidates, not calibrated real-world uncertainty.
• No autonomous control: Do not use this baseline as the sole decision-maker for consequential actions or mission-critical workflows without a human or supervising controller in the loop.

---

8. Public project links

• Dataset: Jev Decisions v1
• Collection: mini-Jev Hugging Face Collection

Future work will continue exploring improved decision models and training on larger real agent/tool-use datasets.

---

Released files and license

• model.safetensors — Trained DecisionHead weights only (262,657 parameters, ~1.1 MB)
• config.json — Architecture specification and backbone reference
• odm_mini.py — Inference loader module
• README.md — Model card and usage documentation
• LICENSE — Apache License 2.0

The mini-Jev decision head and loader code are released under the Apache-2.0 License. The separately downloaded Qwen3-0.6B backbone is governed by its own license terms from the Qwen team.