LightJev-0.6B-v0.1

LightJev-0.6B v0.1

A trained finite-candidate decision prototype built on Qwen3-0.6B by RongxinAI. Full weights, tokenizer, training logs, frozen gold-only data and held-out predictions are included. This is a synthetic-domain research release, not a general business decision model or an official Jev reproduction.

Selected arm: CE, checkpoint step 250, chosen only by development cross-entropy. This selected checkpoint contains 20 head-only plus 230 full-model updates (4,000 sampled questions); each completed training arm ran the longer 20 + 300 update budget (5,120 sampled questions). It has 596,052,993 parameters, including a shared 3,073-parameter LayerNorm/scalar scoring head. Choice, Boolean and ordinal Score all use a candidate softmax. Each candidate carries the complete context through the backbone; there is no autoregressive answer generation, prefix-sharing claim, or RLCD implementation.

Load and predict

This is a custom LightJev scoring checkpoint.

Installation and first run

Use Python 3.10 or newer (Python 3.12 was used for validation):

python -m venv .venv
source .venv/bin/activate
python -m pip install 'git+https://github.com/rongxinzy/[email protected]' huggingface_hub

On Windows, activate with .venv\Scripts\activate. CPU inference works without CUDA. For NVIDIA GPU inference, install a CUDA-enabled PyTorch build compatible with your driver; check python -c "import torch; print(torch.cuda.is_available())" before selecting device="cuda".

The public checkpoint requires no HF token. First download includes about 2.38 GB of FP32 weights, plus tokenizer and metadata; later calls reuse the Hugging Face disk cache. Runtime memory exceeds the weight-file size; no minimum RAM/VRAM requirement has been benchmarked. Save the Python example below as example.py, then run python example.py.

from huggingface_hub import snapshot_download
from lightjev.inference import predict

checkpoint = snapshot_download('rongxinzy/LightJev-0.6B-v0.1')
records = [{
    'id': 'example-1', 'group_id': 'example',
    'state': 'Home log: user wants the dining room fan set to on; access=yes; occupants=2; clock=20:00.',
    'question': 'Select exactly the room and device named in the request. Ignore authorization and occupancy for this question.',
    'kind': 'choice', 'candidates': ['balcony speaker', 'dining room fan'],
}]
print(predict(checkpoint, records, device='cpu'))

This illustration uses the frozen test set's first in-domain question, with its target removed; it is not an additional generalization test.

Use device='cuda' on a supported GPU. The default uses uncalibrated scores (temperature=1). The separately fitted global temperature is 0.818731; opt in with temperature=0.8187307530779818 only after checking suitability for your own distribution. This fit is not a correctness guarantee. Inputs longer than 256 tokenizer tokens per candidate raise an error; no silent truncation. Score expectation means ordinal candidate index, not a numeric value parsed from text.

Input, output and multiple questions

Every input record needs nonempty string fields id, group_id, state, question, plus kind and candidates. target is optional for inference; omit it for your own requests. Candidates must be 2–255 unique, nonempty strings.

predict() returns a list in input order. The example's verified CPU result selects dining room fan, with probabilities approximately [0.000000076, 0.99999988]; small numerical differences across devices are expected. Each result contains id, kind, candidates, probabilities, selected, zero-based selected_index, and temperature.

Pass several records in one predict(checkpoint, records, device="cuda") call to share model loading. Each call loads the model again, and records inside a call are processed sequentially; this is not a persistent server or a cross-question GPU batching API. The download cache avoids downloading again, not loading weights into memory again. A long-running service needs a wrapper that retains load_checkpoint()'s model/tokenizer and implements the same encoding, scoring and per-question softmax.

The 256-token limit applies to the entire formatted input for each candidate, including state, question, candidate and prompt text. Shorten oversized inputs explicitly. Default temperature=1 is unchanged; the released fitted temperature worsened the selected model's held-out CE/ECE.

Command-line inference with JSONL

After the Python example has defined checkpoint and records, save them locally:

import json
from pathlib import Path

Path("checkpoint-path.txt").write_text(checkpoint, encoding="utf-8")
Path("input.jsonl").write_text(
    "".join(json.dumps(row, ensure_ascii=False) + "\n" for row in records),
    encoding="utf-8",
)

On a POSIX shell:

lightjev predict --checkpoint "$(cat checkpoint-path.txt)" \
  --input input.jsonl --output predictions.json --device cpu

Use --device cuda for a compatible GPU. Input is one JSON object per line with unique IDs; output is a JSON array. For offline inference, use the previously downloaded local checkpoint path; the loader requires manifest.json, model.safetensors, backbone/ and tokenizer/, and does not fetch missing files.

Backend status: the published interface uses PyTorch/Transformers. Direct vllm serve loading is not implemented or validated. Do not load these scoring weights through AutoModelForCausalLM or a chat-completions API.

Training

• Base: Qwen/Qwen3-0.6B, revision c1899de289a04d12100db370d81485cdf75e47ca.
• Two independent arms, CE and vector Brier, same seed 17 and data order. Both ran 20 head-only optimizer steps plus 300 full-parameter steps, effective batch 16 (8 x accumulation 2), 5,120 sampled questions with replacement.
• Backbone LR 2e-5, head LR 2e-4, head-warmup LR 1e-3; AdamW, gradient clipping 1, gradient checkpointing. FP32 parameters/Adam moments with BF16 autocast. One NVIDIA RTX 6000D per arm; two arms ran concurrently.
• Development CE selects checkpoints at initialization, head boundary, every 50 global steps and final step. The same development rule selects the published arm. Temperature uses calibration only. Final evaluation reloads full weights and uses FP32.
• Environment: Python 3.12.3, PyTorch 2.13.0+cu130, Transformers 5.16.0.dev0, safetensors 0.8.0.
• Seeds fixed; GPU kernels can be nondeterministic. One seed and one short budget do not establish superiority over NanoJev or Jev.

Reproduce using the training protocol. Exact configuration, selected steps, losses, data hashes and step histories are in manifest.json, release.json and training/. Two early BF16-master trials were stopped to correct parameter precision; their logs are retained and excluded from model selection. Training master parameters in the completed runs and the released weights are FP32.

Data and attribution

Converted from C-Tianyu/NanoJev-Data stage1, revision 87061eb91e8fc687e9b046454afdcc5551e3eff7, source SHA256 7294765b80e751fc5aee7aba906b28a8ea80d6f147253d3f6c3e4f491de0b2d7.

Each of 2,312 source records declares CC0-1.0 and self-authored programmatic provenance. We rely on these per-record declarations, not a blanket license inference for the source repository. Only deterministic-truth and exact conditional-distribution gold labels were retained; teacher payloads, optimal-action-policy targets and missing-kind questions were excluded before training. Thank you to C-Tianyu/TianyuCodings for the programmatic data and NanoJev research direction. LightJev code is independently implemented; it does not copy NanoJev weights or implementation.

The frozen exported data/ has train 3,200 / dev 424 / calibration 424 / test 848 / OOD 448 questions. Original source-group splits are preserved and audited for cross-split IDs/groups/state IDs and normalized duplicate model inputs. Families cover smart-home rules, catalog lookup, grid and tic-tac-toe state judgments, and known-chance probability tasks. Templates and task families overlap; the source-defined OOD split is a limited synthetic distribution shift, not evidence of broad real-world generalization. The preparation script, full exclusions and hashes are available in the code repository and data/manifest.json. Data in data/ retains CC0-1.0; model/code/base license is Apache-2.0. No teacher records are distributed.

Held-out results

All metrics below come from reloading the saved checkpoint. Test contains 656 hard and 192 soft questions; OOD contains 352 hard and 96 soft questions. For the selected CE model, the fitted temperature worsens held-out CE and ECE; the default remains unscaled T=1 and no calibration improvement is claimed. Hard metrics use one-hot targets; soft L2 uses exact conditional-distribution targets. The subsets can overlap for degenerate conditional distributions. Overall CE includes both kinds. Accuracy does not change under temperature scaling. All evaluation predictions, family breakdowns, reliability bins, selective-risk results and raw/calibrated metrics are under evaluation/.

These are synthetic, single-seed measurements. Neither probability normalization nor temperature fitting certifies calibration in another domain. Keep the weak families and OOD failures visible; do not use this release as an autonomous high-stakes decision system without separate domain validation.

Additional out-of-template probe

A separate hand-written CPU probe failed: state The room temperature is 30 degrees C. Cooling is required above 26 degrees C., question Should cooling be enabled?, candidates false, true. The model selected false, with probabilities approximately [0.523605, 0.476395], although the stated rule requires true. This one probe is not included in the frozen benchmark metrics and illustrates why these synthetic results must not be treated as general natural-language decision capability.

Files

• model.safetensors, backbone/, tokenizer/, manifest.json: complete loadable scoring checkpoint.
• temperature.json: optional calibration fit, never automatically applied.
• training/: completed and aborted arm logs/configuration.
• evaluation/: full predictions and metrics for both completed arms.
• data/: frozen CC0 gold-only transformed splits and provenance manifest.
• SHA256SUMS: hashes of release files.

Inspired by TypeSafe AI Jev and NanoJev; no affiliation or proprietary-recipe equivalence is claimed.