Downloads · 30 days
11
16% of all-time downloads
kuririrn/qwen2.5-7b-agent-trajectory-lora-constraint_gen-dist_allign
qwen2.5-7b-agent-trajectory-lora-constraint_gen-dist_allign is a text generation model from kuririrn. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This repository provides a LoRA adapter fine-tuned from Qwen/Qwen2.5-7B-Instruct using LoRA + Unsloth.
Downloads · 30 days
11
16% of all-time downloads
All-time downloads
67
Public
Parameters
7.6B
5.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.9 GB · 100%
How the weights are stored.
U86.7B · 86%
From the Hugging Face model README
This repository provides a LoRA adapter fine-tuned from Qwen/Qwen2.5-7B-Instruct using LoRA + Unsloth.
This repository contains LoRA adapter weights only. The base model must be loaded separately.
This adapter is trained to improve multi-turn agent task performance on ALFWorld (household tasks) and DBBench (database operations).
Loss is applied to all assistant turns in the multi-turn trajectory, enabling the model to learn environment observation, action selection, tool use, and recovery from errors.
To mitigate action mismatch issues observed in AgentBench (e.g., invalid actions being replaced by BLEU-based matching), we introduced an additional constraint during training:
When "Admissible actions" are present in the environment observation, the model is explicitly instructed to:
This improves robustness in environments that apply post-processing or candidate-based action matching.
We observed a formatting discrepancy between the SFT dataset and the AgentBench evaluation environment:
Think: / Act: tagsTHOUGHT: / ACTION: tagsTo reduce distribution mismatch and improve action parsing stability, we normalized assistant outputs during training by converting:
Think: → THOUGHT:Act: → ACTION:This alignment improves consistency with the evaluation parser (e.g., regex-based action extraction) and reduces invalid-action rates caused by format inconsistencies.
This modification particularly benefits smaller models (e.g., 4B), which are more sensitive to surface-form distribution shifts.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = "Qwen/Qwen2.5-7B-Instruct"
adapter = "your_id/your-repo"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
base,
torch_dtype=torch.float16,
device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)
Training data: u-10bei/sft_alfworld_trajectory_dataset_v5
Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License. Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.