Downloads · 30 days
6
7% of all-time downloads
ZeterMordio/anchor-negotiation-sdpo-qwen35-2iter-gen96
anchor-negotiation-sdpo-qwen35-2iter-gen96 is a reinforcement learning model from ZeterMordio. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product.
This repository contains the final checkpoint and metrics from a 2-iteration production-shape smoke test of buyer-only negotiation RLVR with SDPO token shaping.
Downloads · 30 days
6
7% of all-time downloads
All-time downloads
82
Public
Parameters
9.4B
37.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors18.8 GB · 100%
From the Hugging Face model README
This repository contains the final checkpoint and metrics from a 2-iteration production-shape smoke test of buyer-only negotiation RLVR with SDPO token shaping.
Qwen/Qwen3.5-9BQwen/Qwen3.5-9B frozen regulated sellerNUM_ITERS=2, BATCH_SIZE=16, GROUP_SIZE=8, 128 episodes/iterationMAX_TURNS=6, MAX_NEW_TOKENS=300NATIVE_THINK_TOKENS=300, NATIVE_FINAL_TOKENS=96SDPO_LAMBDA=0.9 at iter0 and 0.88 at iter1LR=3e-6, WARMUP_STEPS=10, WEIGHT_DECAY=0.01, GRAD_CLIP_NORM=1.0torch.inference_mode() self-teacher forward| Iteration | Mean reward | Deal rate | Buyer format errors | Loss | First-offer ratio | Peak reserved VRAM |
|---|---|---|---|---|---|---|
| 0 | -0.0744 | 46.9% | 12/128 | 0.1787 | 0.820 | 79.8 GB |
| 1 | -0.1196 | 38.3% | 19/128 | 0.1669 | 0.706 | 80.3 GB |
The run completed and pushed iter-1, iter-2, and final main. It early-stopped after two consecutive buyer-format-warning iterations.
Total runtime was 49.2 minutes. Average per iteration:
Update bottlenecks:
Conclusion: the implemented safe optimizations helped the code path but do not change the dominant bottleneck: full-parameter 9B backward plus CPU-state AdamW on a nearly saturated single A100.
This is a smoke checkpoint, not a recommended final policy. Format errors rose from 12/128 to 19/128, and reward/deal rate degraded in the second iteration. For the next objective-preserving run, use a more conservative stability config: lower LR (e.g. 1e-6) or longer warmup, slower SDPO handoff / more GRPO-heavy early iterations, and consider shorter native-thinking budgets only after testing whether they preserve reward.
from transformers import AutoProcessor, AutoModelForImageTextToText
model_id = "ZeterMordio/anchor-negotiation-sdpo-qwen35-2iter-gen96"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
For text-only negotiation use, call the processor/tokenizer with chat-template messages as done in train_negotiation_sdpo.py included in this repository.
This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.