NanoJev-bf16

NanoJev-bf16

A bf16 weight conversion of the root checkpoint of C-Tianyu/NanoJev — a nano replica of TypeSafe's Jev System One decision model: a Qwen3-0.6B backbone (28 layers, hidden 1024) with structured decision heads, 596M parameters in total. It returns complete probability distributions over dynamic candidates (choice 2–255 options, boolean, score 2–10 levels) in one batched backbone forward, with zero output-token decoding.

This repository is a precision variant, not a new model. No training was performed. All tensors were cast fp32 → bfloat16; every other file (config.json, backbone_config/, tokenizer/) is copied byte-for-byte from upstream.

To be precise about the terminology: this is a precision conversion, not a post-training quantization. bfloat16 is a floating-point format (1 sign / 8 exponent / 7 mantissa bits), so no scale factors or zero-points are stored and no calibration data was used — only the mantissa width shrank. Quantization to integer or low-bit grid formats (int8, 4-bit, GGUF-style) is a separate exercise and is not what this repository contains. No fine-tuning was involved either: not a single weight value was updated.

• Upstream checkpoint: root best.safetensors, produced by the upstream run v3_teacher_coords_multi_seed17 (objective teacher, view coords_multi, best_step 1200), sha256 fff62d1412685c1714eaa386acb603f9690371fb3cc8ad03dc41319302597c28
• Upstream code: https://github.com/TianyuCodings/NanoJev (MIT)
• Upstream data: https://huggingface.co/datasets/C-Tianyu/NanoJev-Data
• License: none is claimed by this repository. The upstream model page declares no license field, so the license of these weights is unresolved upstream. This artifact is published as a derivative of the upstream repository; refer to upstream for all rights questions. It is not affiliated with TypeSafe or with Jev.

Files

Drop-in compatibility

The upstream loader (scripts/predict_toy_decisions.py) builds the model in fp32 and calls load_state_dict(strict=True), so this file loads with no code change at all — the weights are simply upcast back to fp32 on load. Measured on 16 frozen dev states (48 questions):

• argmax agreement with the fp32 file: 48/48
• mean total-variation distance between the two distributions: 1.6e-4

Measured effect (upstream frozen stage2/dev.jsonl: 120 states / 360 questions / 898 candidate paths)

Against the upstream teacher targets (the run's own training objective), fp32 vs bf16:

• argmax agreement: 99.72 % (boolean 100 %, score 100 %, choice 99.17 %)
• mean TV distance: 8.9e-4; mean KL in both directions ≈ 5e-6
• teacher-target CE: 0.499212 → 0.499243 (+0.00003)
• gold-target CE: 0.606506 → 0.606342

Our fp32 reproduction on that split gives CE 0.499212 against the upstream published best_dev_target_ce of 0.4991 for this run, with the same 357/360 eligible questions — the evaluation harness is validated against upstream's own number before any delta is quoted.

Precision guidance when running it

Upstream ships a CUDA-only inference entry point, so running on Apple silicon requires a small patch (accept mps/cpu, and cast the weights instead of relying on autocast). The bf16 file itself needs no patch. Measured on an Apple M4 / macOS with torch 2.14.0, same frozen split:

• MPS + bf16 compute: 82.4 ms/path vs 90.5 ms/path for fp32 (about +9 %).
• CPU + bf16 compute: 1511.6 ms/path vs 134.8 ms/path for fp32 — ~11× slower. Apple silicon has no native bf16 vector path, so these matmuls are emulated. On CPU, use this file as a storage format only: fp32 compute reading the bf16 weights (a drop-in swap that halves disk and memory with unchanged numbers).