Downloads · 30 days
10
23% of all-time downloads
Eyerf/agentblackbox-qwen3-8b-rag-repair-ranker-lora
agentblackbox-qwen3-8b-rag-repair-ranker-lora is a text generation model from Eyerf. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
This repository contains a LoRA adapter for a Qwen3-8B pairwise repair ranker used in AgentBlackBox.
Downloads · 30 days
10
23% of all-time downloads
All-time downloads
44
Public
Repo size
361 MB
Likes
0
Public
Click a slice to open those files.
.safetensors349 MB · 97%
From the Hugging Face model README
This repository contains a LoRA adapter for a Qwen3-8B pairwise repair ranker used in AgentBlackBox.
The model compares two candidate repairs for a failed RAG agent trace and predicts which repair is more likely to resolve the failure under counterfactual replay.
Qwen/Qwen3-8BThis repository contains only the LoRA adapter, not the full base model. Users must load it together with the compatible base model and follow the base model license and terms.
This adapter is intended for research on language-agent debugging, repair selection, and counterfactual replay analysis.
It is designed for AgentBlackBox-style repair ranking:
failed agent trace + repair candidate A + repair candidate B -> preferred repair
Evaluation is performed on replay-labeled RAG repair candidates from AgentBlackBox.
| Split | Top-1 | Top-2 | MRR | NDCG | AUROC | Oracle upper bound |
|---|---|---|---|---|---|---|
| dev | 0.6080 | 0.6751 | 0.6533 | 0.6668 | 0.5454 | 0.7149 |
| test | 0.6873 | 0.7059 | 0.6937 | 0.7086 | 0.5624 | 0.7647 |
On the test split, the pairwise ranker improves over raw candidate order and repair confidence. Diagnosis-conditioned structural priors and XGBoost remain strong competitive baselines.
With AgentBlackBox installed:
python -m agentblackbox.cli rank \
examples/case_studies/rag_mixed_trace_qwen_success/trace_cli.json \
examples/case_studies/rag_mixed_trace_qwen_success/repairs_cli.jsonl \
--ranker qwen_pairwise \
--model-name-or-path Qwen/Qwen3-8B \
--adapter-dir Eyerf/agentblackbox-qwen3-8b-rag-repair-ranker-lora \
--aggregation log_odds \
--output ranking.json
Please cite the AgentBlackBox project if this adapter is useful for your research.
The replay-labeled RAG repair outcome dataset is available on HuggingFace:
Eyerf/agentblackbox-rag-repair-outcomes
It includes failed RAG traces, candidate repairs, counterfactual replay labels, pairwise repair-ranking data, evaluation outputs, and final result tables.