Ouro-1.4B-Thinking-terminal-sft

Ouro-1.4B-Thinking — terminal-agent SFT

Full-backbone supervised finetune of ByteDance/Ouro-1.4B-Thinking on a mixture of public terminal-agent trajectory corpora, aimed at Terminal-Bench.

Why

Evaluated with the terminus-2 agent harness, the base Thinking model scored 0/15 tasks — but the dominant failure was format, not shell knowledge: it reasons in prose for hundreds of tokens and never emits the JSON action object the harness parses into commands, so the agent loop cannot extract a single action. This finetune targets that specific gap.

Same prompt, both models driven through the same decode loop:

Training data

Streaming interleave of three corpora, all generated by driving a strong teacher model through the same terminus-2 harness Terminal-Bench uses, so the assistant turns already carry the target action format:

Trajectories are truncated to their first 64 turns, and loss is masked to assistant turns only (terminal output fed back to the agent contributes no gradient).

Training

Loss moved 0.6→0.4–0.7 with grad-norm falling 22→3–6. Weights shifted 1.2–3.4% relative to the base (lm_head most, at 3.4%).

The Stage-I objective trains every one of the 4 recurrence depths, weighted by the exit distribution, rather than only the blended output — this preserves the adaptive-computation behaviour the architecture is built around.

Status / limitations

• A full Terminal-Bench run against this checkpoint has not been completed yet. The comparison above is a single-prompt check, not a benchmark score. Treat it as evidence the format problem is fixed, not as a task-success claim.
• 1000 steps over a mixture this size is a small fraction of one epoch.
• Trained on terminus-2-style prompts; other agent harnesses with different response schemas are out of distribution.

Loading

Ouro is a custom architecture (trust_remote_code=True). This repo bakes in a pad_token_id default the released config omits, but recent transformers versions still need shims for Ouro's bundled modeling code (a RoPE-init fallback and attention-mask signature adaptation) — these affect the base model identically. See _load_hf_generate_model in torchtitan's scripts/evaluate_humaneval_evalplus.py for a working reference loader.

model.generate() is not a supported path: Ouro's cache expects a cache_position tensor that generate() does not supply. Use a token-by-token decode loop that calls model.model(...) directly (see generate_chat in scripts/ouro_openai_server.py).