Downloads · 30 days
7
22% of all-time downloads
MANGSEOK123/qwen3-4b-tau2-oel-retail-ep1
qwen3-4b-tau2-oel-retail-ep1 is a machine learning model from MANGSEOK123. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Qwen3-4B-Instruct-2507 after one epoch of Online Experiential Learning (OEL) on the tau2-bench retail domain.
Downloads · 30 days
7
22% of all-time downloads
All-time downloads
32
Public
Parameters
4.4B
8.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8.8 GB · 100%
From the Hugging Face model README
Qwen3-4B-Instruct-2507 after one epoch of Online Experiential Learning (OEL) on the tau2-bench retail domain.
OEL here is self-distillation, not RL — no reward signal is used. Teacher and student share weights; the only difference is that the teacher gets the task's memory appended to its system prompt. The student is fitted to the teacher's distribution with a token-level full KL loss, so that the behaviour survives without the memory in the prompt.
deploy the student plays each task WITHOUT memory; every agent turn is pickled
consolidate the same trajectories are re-scored with each task's memory injected
(the teacher), and the memory-free student is fit to that distribution
merge FSDP shards -> HuggingFace weights
| base model | Qwen3-4B-Instruct-2507 |
| domain | tau2-bench retail |
| data | 104 (memory, synthesized task) pairs |
| batch size | 8 (4 GPUs) |
| steps | 13 (1 epoch, last batch 8) |
| learning rate | 3e-6 |
| KL loss | full, over every response token |
| grad clip | 1.0 |
| user simulator | gpt-4.1-mini |
Evaluated with tau2-bench's own CLI (tau2 run), retail test split, 40 tasks × 3
trials, greedy agent, no memory in the prompt — the memory the model was trained
against never reaches the test prompt.
| metric | base | this model |
|---|---|---|
| avg@3 | 0.400 | 0.392 |
| pass@3 | 0.575 | 0.550 |
| pass^3 | 0.275 | 0.225 |
This checkpoint does not improve over its base model. All three differences are under one standard error (SE ≈ 0.064–0.078 at 40 tasks), so they are indistinguishable from noise rather than a demonstrated regression — but there is no evidence of a gain either.
This is consistent with what the method's authors report: across roughly twenty runs over three domains, several memory sources, 1–2 epochs and learning rates from 1e-6 to 1e-5, no configuration beat its base model by more than about 1.5 SE. Putting the same memories in the prompt at test time did clearly help, so the memories carry signal; the distillation step recovers only part of it.
Published as a reproducible artifact of that negative result, not as an improved model.
Gradient norms ran 0.6–14.7 against a clip of 1.0, so the effective step size was set by the clip rather than by the learning rate. Only the final step is saved; its entropy here was 0.329, in the healthy range by the authors' measurements — yet that did not translate into better task performance, which suggests entropy at the last step is a weak proxy for checkpoint quality.
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("MANGSEOK123/qwen3-4b-tau2-oel-retail-ep1")
t = AutoTokenizer.from_pretrained("MANGSEOK123/qwen3-4b-tau2-oel-retail-ep1")
Serving for tau2 evaluation needs hermes tool calling:
vllm serve MANGSEOK123/qwen3-4b-tau2-oel-retail-ep1 \
--enable-auto-tool-choice --tool-call-parser hermes