Hemmingway-1-NInfer

Hemmingway-1 — NInfer v3 artifact

Exploration release. Published to demonstrate that a third-party finetune can be quantized into the NInfer v3 format with the upstream groupwise-int recipe. It is not part of this publisher's benchmarked model line: quality suites and independent kit-verification are deliberately not planned (see Quality). Use it freely; do not read it as a vetted release.

Single-file NInfer v3 artifact of Altworld/Hemmingway-1 (an everyday-writing finetune of Qwen3.8-27B), re-containerized for the NInfer v3 runtime with the MTP head, DFlash2 drafter and the indexed proposal head assembled in.

This is a quantization/re-containerization, not a new training run. All stored codes are computed at conversion time from the pinned BF16 finetune by the upstream qwen3_8_27b groupwise-int recipe. See Composition.

Composition

Conversion used the upstream NInfer v3 converter (tools/convert) with the built-in qwen3_8_27b recipe — the same recipe that produces the official neroued/Qwen3.8-27B-NInfer artifact (revision 1cbd84e7221e51186bd7f093a149912d2489625b). Projections are quantized with per-group max-abs groupwise quantization; activation policy is A16Only throughout (weight-only quantization).

Stored object counts: 851 tensors in 855 objects — q4_g64_fp16 129, q5_g64_fp16 192, q8_g32_fp16 28, bf16 405, fp32 96, int32 1.

Tokenizer note

Hemmingway-1's published tokenizer.json declares the older Qwen2 pre-tokenizer ([\p{L}]+) and omits added_tokens_decoder; the NInfer Qwen frontend validates one exact pipeline and refuses to load it. The artifact therefore embeds the upstream Qwen3.8 tokenizer resource declaration. This is a declaration fix, not a vocabulary swap: the two tokenizers have byte-identical vocab (248,044 entries), merges (247,587) and added_tokens (33), and derive the same token domain (248,077 tokens, same 21 special ids). The finetune's own generation_config.json is preserved, so stop-token semantics are unchanged.

Serving

Requires an NInfer v3 runtime — the upstream Neroued/ninfer engine serves this artifact as-is. 64-bit Linux, CUDA toolkit with sm_120a, one RTX 5090 (32 GB).

Tested engine revision: Neroued/ninfer @ 5b4303c0ea0e8ab2be3efa54a677829f3edab6e5. This artifact was built and validated against that commit; use it, or a revision whose conversion and kernel paths are known-compatible with it. The artifact format is not versioned against later engine releases, so a newer upstream revision is not guaranteed to load or execute this file — pin the engine, not just the artifact.

git clone https://github.com/Neroued/ninfer
cd ninfer && git checkout 5b4303c0ea0e8ab2be3efa54a677829f3edab6e5

# native MTP=3 speculation, native 262K context
ninfer-serve hemmingway_1.ninfer \
  --model-id hemmingway-1 \
  --spec mtp --draft-tokens 3 --lm-head-draft \
  --max-context 262144 --kv-dtype nvfp4 --max-concurrency 3 \
  --host 0.0.0.0 --port 30000

# alternatively the DFlash2 companion embedded in the same file
ninfer-serve hemmingway_1.ninfer \
  --spec dflash2 --draft-tokens 7 --lm-head-draft \
  --max-context 262144 --kv-dtype nvfp4

Do not pass --vision: this artifact contains no vision component, and the loading phase fails if one is requested.

Memory, measured on a single RTX 5090 32 GB with NVFP4 KV and native 262K context (auto-sized by the engine):

Because there is no vision tower, the whole fixed allocation goes to KV — the pool is larger than the vision-enabled Qwen3.8-27B artifacts on the same card.

Quality

Not benchmarked by design. This artifact is an exploration release: the goal was to test whether a third-party finetune can be quantized into the NInfer format with the upstream groupwise-int recipe. That question is answered by the artifact loading and serving correctly, which it does. The publisher's standard evaluation suites (AIME / GPQA-Diamond / IFBench / NIAH, and the upstream serving-corpus draft-acceptance method) are deliberately not planned for this release, and no independent kit-verification run is claimed.

Treat every number below as single-request serving smoke results, not benchmark scores.

A quality section will be added once the benchmark suites have been run.

Lineage and attribution

Qwen/Qwen3.8-27B                        (BF16 base, Apache-2.0)
  └─ Altworld/Hemmingway-1              (finetune, Apache-2.0)
       ├─ z-lab/Qwen3.8-27B-DFlash2     (DFlash2 drafter, Apache-2.0)
       └─ Neroued/ninfer @ 5b4303c0     (v3 engine, converter, artifact format)

• Hemmingway-1 is by Altworld; its card describes it as built for everyday writing (messages, emails) and reports its own CommunicationBench / Human-Likeness results. Those numbers belong to the finetune and are unaffected by this re-containerization.
• DFlash2 is by Inco AI (block-diffusion drafter, ICML 2026); this artifact embeds the drafter weights quantized to W8G32 as the speculative companion.
• The v3 artifact format, converter and all kernel paths are from Neroued/ninfer (upstream).
• This repository is an independent packaging of these weights for the upstream runtime; it is not an official release by the upstream engine maintainers, by Altworld, or by Inco AI.

Provenance notes

Every external input is pinned to an immutable revision. Source weights were downloaded and verified file-by-file against their published SHA-256 digests.

Embedded resource digests: tokenizer.json = 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3, tokenizer_config.json = b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27.

Conversion: recipe qwen3_8_27b, --device cpu, components text,mtp,dflash2, --proposal (131,072 rows), chat template tools/chat_templates/qwen3_8.jinja. Wall time 1,131 s. hemmingway_1.ninfer.conversion.json (shipped alongside) records every per-tensor method, format, source and layout chosen by the run.

Reproduction kit: conversion/ in this repository.

Caveats

• The MTP head is the finetune's own (trained by Altworld); the DFlash2 companion is a separate drafter that was trained against the Qwen3.8-27B base, not against this finetune. Expect its acceptance rate to be lower than on the base model — on this artifact it measured ~23–29% against ~53% for the artifact's own MTP head. Prefer MTP unless you measure otherwise.
• Serving numbers above come from one RTX 5090 and the upstream engine revision 5b4303c0; they are reproducibility anchors, not vendor-published figures.
• Text only: no image or video input.