adaptive-qwen3-0.6b-tool-result-router-r4-step600

Adaptive Qwen3-0.6B Tool-Result Router (LoRA r=4, step 600)

This repository contains the selected step-600 research checkpoint of an adaptive router for historical tool-result segments. The router predicts whether each segment can be omitted from an agent's context.

This is not a standalone language model checkpoint. It contains:

• adapter/: a PEFT LoRA adapter for Qwen/Qwen3-0.6B;
• classifier.pt: the binary segment-classification head.

The base model must be loaded separately.

Task

The router jointly classifies the segments within one tool result. Its causal input includes an instruction, the current user request, same-turn tool context, numbered segment texts, and one classification slot per segment. The classifier reads the hidden state at each classification slot.

The binary target is:

• 1: DISCARDABLE;
• 0: USED_IN_FUTURE or POTENTIALLY_USEFUL.

Ambiguous labels were excluded from router loss. At inference time, uncertain segments should use a conservative retain/compress fallback rather than automatic omission.

Architecture and training

• Base model: Qwen/Qwen3-0.6B
• Context length used for this stage: 32,768 tokens
• Adaptation: LoRA
• LoRA rank: 4
• LoRA alpha: 8
• LoRA dropout: 0.05
• Target modules: attention and MLP projection layers
• Classification head: hidden size 256, binary output
• Training data: labeled APIGen-MT and AReaL-tau2 tool-result trajectories
• BFCL labels were kept external during generalist training

Older complete interactions are removed as structured units when necessary to fit the context window; labeled segment text is not silently truncated.

Loading

The LoRA adapter can be loaded with PEFT:

from transformers import AutoModel
from peft import PeftModel

base = AutoModel.from_pretrained(
    "Qwen/Qwen3-0.6B",
    torch_dtype="auto",
)
model = PeftModel.from_pretrained(base, "Graph-COM/adaptive-qwen3-0.6b-tool-result-router-r4-step600/adapter")

classifier.pt is a separate PyTorch state dictionary and requires the router architecture and input formatting implemented in the Latent-Agent research codebase:

• data/adaptive_qwen_router.py
• data/adaptive_qwen_router_model.py
• scripts/eval_adaptive_qwen_router_ddp.py

Intended use

This checkpoint is intended for research on adaptive context compression in tool-using agents. It was designed to route historical tool-result segments, while tool schemas remain uniformly compressed.

The checkpoint should not be treated as a general-purpose safety classifier. The omission threshold must be calibrated for the target domain. A false discard can remove information needed by a later action, so conservative fallback behavior is recommended.

Limitations

• Labels were produced by a model-assisted cascade and remain partly subjective.
• Domain transfer can shift router calibration.
• The adapter and classifier require the matching custom prompt construction and classification-slot extraction.
• This repository does not include the base Qwen weights, training datasets, or evaluation trajectories.