modernbert-ja-70m-aizuchi

modernbert-ja-70m-aizuchi

A small Japanese classifier that decides which kind of backchannel (相槌, aizuchi) a listener should give first to what the user is saying, from the assistant's previous utterance and the user's (possibly unfinished) speech-recognition transcript. It also predicts whether the user has finished the sentence.

It is meant for voice assistants and spoken-dialogue systems that want to react within a few hundred milliseconds, before the main answer is ready: a quick 「うん」 to a reply, 「ちょっと見てみますね」 to a search request, 「え、そうなんですか」 to surprising news, or a small 「うん」 while the user is still in the middle of a sentence.

• Base model: sbintuitions/modernbert-ja-70m
• Format: ONNX (int8 dynamic quantization, 71 MB; fp32 also provided)
• Speed: 1.7 ms per sentence (int8) on an Apple M5 CPU with one thread
• Training data: about 9,000 synthetic Japanese utterances (see Training data)
• Used in: ASIST, a desktop voice assistant

Classes

The classes are cut by the sound the listener makes first, not by what the user did. Two user acts that call for the same sound share a class.

Input and output

Input. The assistant's previous utterance (prev, optional) and the user's transcript (text), encoded as a sentence pair <s> prev </s><s> text </s>, or as text alone when there is no previous utterance. At most 64 tokens; when the input is longer, truncate prev, not text.

Before encoding, normalize both strings the way the training data was normalized: NFKC, then remove whitespace and the characters 、。,.?？!！…・「」『』()（）. Speech recognizers add punctuation inconsistently, and the synthetic data carried punctuation unevenly between classes (check had many 「?」), so the model is trained without it.

Output.

The probabilities are not calibrated. Pick thresholds from the distribution you see in your own application; ASIST plays nothing when the top probability is low, because a wrong backchannel is worse than silence.

Usage

import json, re, unicodedata
import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer

repo = "sakasegawa/modernbert-ja-70m-aizuchi"
session = ort.InferenceSession(hf_hub_download(repo, "aizuchi.int8.onnx"))
tokenizer = Tokenizer.from_file(hf_hub_download(repo, "tokenizer.json"))
tokenizer.enable_truncation(64, strategy="only_first")
classes = json.load(open(hf_hub_download(repo, "labels.json")))["classes"]

PUNCT = re.compile(r"[\s、。,.?？!！…・「」『』()（）]")

def normalize(s: str) -> str:
    return PUNCT.sub("", unicodedata.normalize("NFKC", s))

def classify(text: str, prev: str | None = None):
    text = normalize(text)
    encoding = tokenizer.encode(normalize(prev), text) if prev else tokenizer.encode(text)
    feed = {
        "input_ids": np.array([encoding.ids], dtype=np.int64),
        "attention_mask": np.array([encoding.attention_mask], dtype=np.int64),
    }
    cls_logits, complete_logit = session.run(None, feed)
    p = np.exp(cls_logits[0] - cls_logits[0].max())
    p /= p.sum()
    return classes[int(p.argmax())], float(p.max()), float(1 / (1 + np.exp(-complete_logit[0])))

print(classify("今日の予定を確認して"))
print(classify("それじゃなくてもう一つのほう", prev="会議を10時に入れました。"))
print(classify("あのさ、さっきの話なんだけど"))

In a live system, classify each partial transcript as it arrives and keep only the newest request; one sentence takes under 2 ms on a CPU.

Batches work as well. The fp32 model gives the same probabilities in a padded batch as one sentence at a time. The int8 model quantizes activations per run, so its probabilities move by up to about 0.02 depending on what else is in the batch; the predicted class did not change on the 400 sentences we compared.

Training data

The training data is synthetic. For each of 20 situations a voice assistant meets (calendar, mail, weather, coding, small talk, health, travel and so on), Claude (claude-fable-5-1 and claude-opus-5-5) wrote about 40 utterances per class in the style of speech-recognition output: few punctuation marks, dropped particles, fillers, restarts, half polite and half casual. About a fifth of each class were written as boundary cases that look like a neighbouring class. No real conversation logs were used. After removing duplicates that differ only in punctuation, 9,045 rows remain.

none and flow are smaller because their short set phrases repeat across situations and were removed as duplicates.

The data is published as sakasegawa/aizuchi-synthetic-ja under its own terms of use.

Use of Claude's outputs

The training data was written with Claude Code under a Claude Max plan, so Anthropic's Consumer Terms apply (Claude Code legal and compliance). They forbid using the services to develop products or services that compete with Anthropic's, including training AI models, and the Usage Policy forbids training models on outputs without authorization in the sense of model scraping and distillation.

Anthropic's help article Can I use my Outputs to train an AI model? (updated 2026-03-16) explains where that line is. It allows training models that do not compete with Anthropic's own models and names specialized classifiers, among them content categorization, as examples. It forbids general-purpose chatbots, models for open-ended text generation, using outputs as training targets, and supporting a third party who tries to do those things.

This model is a 70M-parameter encoder that sorts an utterance into 13 classes and returns two probabilities. It generates no text and cannot answer a question, so it falls under the specialized classifiers the help article allows. This is the author's reading of Anthropic's terms as of 2026-10-12, not legal advice.

Training

The encoder's <s> vector feeds two linear heads, one for the class and one for completeness, trained together. Two augmentations:

• prev is dropped with probability 0.3, so that the model also works without the previous utterance.
• With probability 0.6 the user's text is cut at a token boundary, and the cut row is labelled as not complete. When the cut keeps less than 40% of the text or fewer than 4 characters, no class loss is applied, so that a fragment such as 「今日の」 is not taught as work.

The released model was trained on all 20 situations for 4 epochs. The training code is at nyosegawa/modernbert-ja-70m-aizuchi.

Evaluation

All numbers are on synthetic data; there is no evaluation on real conversations yet.

The released model scores lower because its random validation set includes harder situations such as small talk and coding. Most remaining errors are between cheer and surprise, empathy and surprise, and check and understand, where people also disagree, and some synthetic rows carry debatable labels themselves.

Limitations

• Trained only on synthetic, assistant-directed Japanese. Speech with background noise, speech directed at other people, dialects and very long monologues are not covered.
• The classes reflect one design of a voice assistant's first response. If your system needs different sounds, the classes may need to be merged or redefined.
• The probabilities are not calibrated.
• The completeness head is weak on sentences cut early (AUC near chance at 50% and 75% cuts on the released model's validation set); use it together with silence length, not alone.

License

MIT, the same as the base model.