text-gpt-live

Text GPT-Live

Text GPT-Live is an 8-bit MLX checkpoint for a small text interaction model. It continuously evaluates a stream of typed-text snapshots and emits a single next interaction action: stay silent, respond, highlight, suggest an edit, commit a translation unit, start a web search, or delegate background work.

Write-up: Can you train a GPT-Live for $50? · Code: github.com/huyxdang/Text-GPT-Live · Data: huyxdang/text-gpt-live-dataset

Base model and training

• Base: Qwen/Qwen3.5-4B
• Fine-tuning: supervised LoRA, merged into the released checkpoint
• Main g1-v2 stage: 2,903 synthetic decision cards, one epoch
• Delivery-repair continuation: 544 cards, one epoch
• Deployment format: 8-bit MLX affine quantization (group size 64)

The release uses the selected fixed-index delivery repair. It was chosen because the 4-bit candidate changed a required search-result target index in local acceptance, while this 8-bit checkpoint preserved the selected model output.

Behavior and limitations

The model was evaluated on synthetic event streams and local closed-loop demos for dialog/silence, simple highlighting, incremental Chinese translation, and concurrent delegation plus web-search delivery. Demo 5 (time awareness) is not included. This checkpoint is a research/demo artifact, not a general assistant.

Its selected local persistent-cache benchmark measured 971.55 ms median and 1,128.68 ms p95 per model decision; it does not meet the project's 650 ms target. Simple highlighting is also not robust on longer streams, and translation quality remains a known limitation.

Files

model.safetensors is the complete merged 8-bit MLX checkpoint. The repository also includes the selected source LoRA adapter (adapter_model.safetensors and adapter_config.json), so the merge can be reproduced from Qwen/Qwen3.5-4B. MERGE_PROVENANCE.json records the selected merge settings.

Training data

The two selected training stages are published at huyxdang/text-gpt-live-dataset.

License

This release follows the licensing and use terms of its Qwen base model. Review the base model's license before use or redistribution.