ingot-chrono-qwen3-0.6b

Ingot — Chrono SLM (Qwen3-0.6B)

Turns a Japanese or English scheduling sentence into a strict RFC 5545 object: an RRULE, an ISO-8601 local start, an IANA timezone, a duration, exception dates, and a holiday-avoidance policy.

Trained entirely on NagaYu/ingot-chrono, whose labels were constructed before their sentences existed. Code: github.com/NagaYu/ingot.

Read this before using the model

A hand-written dateutil parser beats this model on every split — 90.5% vs 67.5% exact match on test, and the gap is statistically significant (McNemar p = 2.5e-05). If your inputs resemble this dataset's surfaces, use the parser (baseline_rule.py), not this model.

The checkpoint is also deliberately undertrained: 1,000 optimiser steps, roughly 8% of one epoch, stopped so the GPU could run the evaluation inside one session. Validation loss was still falling monotonically (0.042 → 0.024 → 0.013 at steps 200/400/800).

It is published as a reproducible reference point for the Ingot pipeline, not as a recommended parser. The honest artifact of this project is the dataset.

Files in this repository

The bf16 fused checkpoint (1.1 GB) and the Q8_0 GGUF (624 MB) are not uploaded — both are reconstructible from the adapter in about a minute:

python -m mlx_lm fuse --model Qwen/Qwen3-0.6B --adapter-path adapter --save-path fused

Usage

from mlx_lm import load, generate

model, tokenizer = load("NagaYu/ingot-chrono-qwen3-0.6b")

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},   # ingot.prompts.SYSTEM_PROMPT
    {"role": "user", "content": "Today: 2026-04-01 (Wednesday)\nDefault timezone: Asia/Tokyo\n\n"
                                "毎月第2水曜10:30から45分、祝日なら前営業日に"},
]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False, tokenize=False
)
print(generate(model, tokenizer, prompt, max_tokens=160))

{"dtstart":"2026-04-08T10:30:00","tzid":"Asia/Tokyo","rrule":"FREQ=MONTHLY;BYDAY=2WE","duration_minutes":45,"exdate":[],"holiday":{"calendar":"JP","shift":"before"}}

enable_thinking=False is required. Qwen3 is a hybrid-thinking model. That flag makes the generation prefix end with an empty <think></think> block, exactly matching the training rows. Leave thinking on and the model emits a monologue instead of JSON.

reference_date and default_tz are supplied in the prompt because the sentence cannot determine them. Without them, relative expressions ("来週頭", "next Monday") would be unanswerable rather than hard.

Grammar-constrained decoding (GGUF)

python -c "from ingot.decode import save_gbnf; save_gbnf('schedule.gbnf')"
llama-cli -m gguf/ingot-chrono-Q4_K_M.gguf --grammar-file schedule.gbnf -p "$PROMPT"

Evaluation

n = 200 per split, greedy decoding, exact match on RRULE + DTSTART + TZID after canonicalising every system's output. (A) is the hand-written parser, (D) this model at bf16, (E) the Q4_K_M GGUF under grammar constraints. Frontier-API baselines could not be run (no credential) and are reported as n/a, never as zero.

Two things worth more than the headline column:

Semantic accuracy is much higher than exact match. On test, (D) scores 67.5% exact but 83.5% occurrence-exact — in a sixth of cases it writes a rule that is not byte-identical to the gold RRULE yet produces the same next ten meetings. Exact match is the strict metric; occurrence match is the one a calendar user would feel.

The two approaches fail on opposite languages. On unseen_combo:

That is the one slice where they look complementary rather than ranked.

Syntax validity

Measuring this found a real bug: the grammar admitted FREQ=WEEKLY;BYDAY=1TU, an ordinal weekday under WEEKLY, which RFC 5545 forbids. All grammar dialects now couple the ordinal to the frequency, and a test pins the invariant that the grammar must be a subset of valid specs.

Speed (Apple M2, 16 GB, real ~250-token prompt, batch 1)

Training

python scripts/build_dataset.py --n 24000 --variants 2
python scripts/train_lora.py --base Qwen/Qwen3-0.6B --iters 6000   # longer than the release
python scripts/merge_and_quantize.py --model runs/qwen3-0.6b-chrono

Limitations

• Synthetic surfaces. Sentences come from 31 templates plus label-preserving noise operators. Transfer to open-world scheduling text is untested and not claimed.
• Undertrained, and beaten by a rule parser — see the box at the top.
• Quantization is not free, and costs most out of distribution (7.5 points on unseen_template). The repository's quantization budget test fails on this, on purpose.
• Two holiday calendars only (JP, US); years 2026–2028; the Japanese calendar excludes the one-off 2020–2021 Olympic moves.
• Not a general assistant. It emits one JSON object and nothing else.

Citation

@software{ingot2026,
  title  = {Ingot: inverted generation for strictly-labelled synthetic data},
  author = {NagaYu},
  year   = {2026},
  url    = {https://github.com/NagaYu/ingot}
}

Apache-2.0. The base model carries its own licence.