Downloads · 30 days
0
nemotron-think-history-tokenizer
geodesic-research/nemotron-think-history-tokenizer
nemotron-think-history-tokenizer is a machine learning model from geodesic-research. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Fork of geodesic-research/nemotron-think-tokenizer @ cce18a60d4e17b9a0436706fbdc17b706994270b with an identical encoder and three chat-template edits:
Access
Public
Updated Aug 9, 2026
Repo size
17.1 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json17.1 MB · 100%
At a glance
- Library
- transformers
- Access
- Public
- Created
- Aug 9, 2026
- Updated
- Aug 9, 2026
- SHA
- b6cdae23
Model card
From the Hugging Face model README
nemotron-think-history-tokenizer
Fork of geodesic-research/nemotron-think-tokenizer @ cce18a60d4e17b9a0436706fbdc17b706994270b
with an identical encoder and three chat-template edits:
truncate_history_thinking-default-falseno-stub-for-empty-reasoningno-stub-for-empty-toolcall-content
Net effect: rendering never strips prior-turn reasoning (truncate_history_thinking
defaults to False) and never emits an empty <think></think> stub for assistant
messages without reasoning. The {% generation %} loss-mask markers are unchanged.
Built 2026-08-09 21:57 UTC by scripts/data/build_think_history_tokenizer.py (geodesic-megatron).
- Library
- transformers
- Created
- Aug 9, 2026
- Updated
- Aug 9, 2026