Ouro-1.4B-Thinking-terminal-sft-10k

Ouro-1.4B-Thinking — terminal-agent SFT (10k steps)

Full-backbone supervised finetune of ByteDance/Ouro-1.4B-Thinking on a mixture of public terminal-agent trajectory corpora, targeting Terminal-Bench.

10k-step sibling of jaslee/Ouro-1.4B-Thinking-terminal-sft (1k steps).

What this checkpoint does and does not do

Does: makes the model usable as an agent under a wall-clock budget. The base model reasons in prose for ~1,500 tokens per turn (max observed 5,527) and frequently never emits a parseable action. This checkpoint answers in ~150 tokens and stops.

Measured on 12 Terminal-Bench tasks x 2 attempts, identical decode settings (temperature 0.7, no n-gram guard), each model on its own GPU:

Base did not complete a single request: at ~10 tok/s its turns are long enough to exceed the client timeout. Under a lighter node load an earlier run had base working and tied with the 1k SFT (pass@5 0.400 each), so read the table above as "robust to time pressure", not "better at tasks".

Does not: improve task success. Against the 1k-step checkpoint on the same 12 tasks:

Identical tasks; the pass@1 gap is one trial in 24. 10x the training moved training loss 0.72 -> 0.55 and did not move task success. The ten unsolved tasks fail on agent timeout with partial subtest credit — the model produces valid actions and makes real progress, then cannot close out multi-step tasks. That looks like an agentic-competence limit at 1.4B rather than a data-volume or formatting one.

Training data

Streaming interleave of three corpora, all generated by driving a strong teacher through the same terminus-2 harness Terminal-Bench uses, so assistant turns already carry the target JSON action format:

Trajectories truncated to their first 64 turns; loss masked to assistant turns only. allenai/TMax-SFT-16.5K and camel-ai/seta-env were excluded — despite the naming, both are task/environment definitions (pytest validators, container defs), not chat trajectories.

Training

Mean loss by phase: 0.721 (1–1k) → 0.603 (3k–6k) → 0.547 (6k–10k); grad-norm 22 → ~2.

Single GPU is not a preference: 2-GPU FSDP fails on this node with an NCCL ALLGATHER collective timeout.

Loading

Ouro is a custom architecture (trust_remote_code=True). This repo bakes in the pad_token_id the released config omits, but recent transformers still needs shims for the bundled modeling code (RoPE-init fallback, attention-mask signature) — these affect the base model identically. See _load_hf_generate_model in torchtitan's scripts/evaluate_humaneval_evalplus.py.

model.generate() is not supported: Ouro's cache expects a cache_position tensor generate() does not supply. Drive a token-by-token loop calling model.model(...) — see generate_chat in scripts/ouro_openai_server.py.

Do not enable a no-repeat-ngram guard for JSON output. Valid JSON requires repeated trigrams ("duration": 0.1, }, {); banning them corrupts the structure. With no_repeat_ngram_size=3 under sampling this model scored pass@5 = 0.000 purely from malformed JSON; at 0 it scored 0.400 on the same tasks.